A groundbreaking study published in Nature Communications on December 7, 2023, showcases a major advancement in neural network technology, with significant implications for medical diagnostics. A collaborative team, including researchers from the Centre for Nanoscience and Nanotechnology (C2N) at CNRS, CEA-Leti, CEA-List, and IM2NP, has successfully implemented the first complete memristor-based Bayesian neural network tailored for a real-world application. This innovative technology offers enhanced capabilities in classifying different types of arrhythmia recordings, integrating a sophisticated approach to managing uncertainty. This feature is particularly valuable in medical diagnosis and other safety-critical tasks that require accurate decision-making based on limited, often noisy data. Bayesian neural networks are uniquely suited to these tasks due to their ability to assess predictive uncertainty. However, these networks traditionally demand high energy and computational resources due to their probabilistic nature, which necessitates storing probability distributions – or synaptic weights – often using a random number generator.
The research team, including Damien Querlioz from C2N, has innovatively used the intrinsic variability of memristors to store these probability distributions, bypassing the need for separate random number generators. This approach not only reduces energy consumption but also streamlines the overall process.
A significant challenge in implementing this system was the need for extensive parallel multiply-and-accumulate (MAC) operations, which are typically resource-intensive when performed on CMOS-based systems. The team's solution involves using memristor crossbars, which naturally facilitate these operations through Ohm's law and Kirchhoff's current law, leading to a marked reduction in power consumption.
An additional aspect of this research was the integration of memristors with Bayesian neural networks. Memristors, governed by the laws of device physics, often display statistical effects that can be challenging to reconcile with the more arbitrary nature of effects in Bayesian neural networks. The team developed a novel training algorithm incorporating a 'technological loss' that adapts to the idiosyncrasies of the memristors during the learning phase, ensuring compatibility between the network and the memristor's imperfections.
A key advantage of this Bayesian neural network is its uncertainty quantification capability. Unlike traditional neural networks, which might misclassify unfamiliar inputs with false confidence, a Bayesian neural network can recognize and acknowledge unknown or out-of-distribution situations. This feature is critical in environments where erroneous predictions can have severe consequences, such as in medical diagnostics.
The network's unique approach to synaptic values – treating them as probability distributions rather than precise values – results in outputs that are also probabilistic, offering insights into the network's level of certainty. This represents a significant step forward in the field of neural network technology, promising substantial improvements in medical diagnostics and other critical applications.
References
Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks
Djohan Bonnet1,2, Tifenn Hirtzlin1, Atreya Majumdar2, Thomas Dalgaty1, Eduardo Esmanhotto1, Valentina Meli1, Niccolo Castellani1, Simon Martin1, Jean-François Nodin1, Guillaume Bourgeois1, Jean-Michel Portal3, Damien Querlioz2 & Elisa Vianello1
Nature Communications volume 14, Article number: 7530 (2023)
DOI : https://doi.org/10.1038/s41467-023-43317-9
Affiliations
1Université Grenoble Alpes, CEA, LETI, Grenoble, France
2Université Paris-Saclay, CNRS, Centre de Nanosciences et de Nanotechnologies, Palaiseau, France
3Aix-Marseille Université, CNRS, Institut Matériaux Microélectronique Nanosciences de Provence, Marseille, France
Contact C2N : Damien Querlioz
Figure : Scanning Electron Microscopy image of a filamentary memristor in the back end of line of our hybrid memristor/CMOS process
