QC Benchmarks: A Critical Element to QC Progress

During the recent conference held by the Special Interest Group on Knowledge Discovery and Data Mining held in Singapore, three international public science policy advocacy groups presented a guide, Using Artificial Intelligence to Support Healthcare Decisions, aimed at empowering and educating the public on the growing use of AI platforms in the healthcare decision-making process. The guide covers explanations of common applications of artificial intelligence platforms in healthcare and, more importantly, outlines specific questions one can pose to cut to the core of the efficacy and reliability of an AI platform in those applications.

MLCommons, an international artificial intelligence (AI) standards body formed in 2018, launched MLPerf Tiny, their first benchmark targeted at the inference capabilities of edge and embedded devices, or what they call "intelligence in everyday devices". The new benchmark is now part of the overall MLPerf benchmark suite, which measures AI training and inference performance on a wide variety of workloads, including natural language processing and image recognition. The benchmark covers four machine learning (ML) tasks focused on camera and microphone sensors as inputs: keyword spotting, visual wake words, tiny image classification, and anomaly detection. Some important use cases include smart home security, virtual assistants, and predictive maintenance.