Pre-training and Meta-learning for Memristor Crossbar Arrays
Emre Neftci a
a Forschungszentrum Jülich and RWTH Aachen University, Germany
Proceedings of Neuronics Conference (Neuronics)
València, Spain, 2024 February 21st - 23rd
Organizers: Sabina Spiga and Juan Bisquert
Invited Speaker, Emre Neftci, presentation 027
Publication date: 18th December 2023

Memristive crossbar arrays show promise as non-von Neumann computing technologies, bringing sophisticated neural network processing to the edge and facilitating real-world online learning. However, their deployment for real-world learning problems faces challenges such as non-linearities in conductance updates, variations during operation, fabrication mismatch, conductance drift, and the realities of gradient descent training.
This talk will present methods to pre-train neural networks to be largely insensitive to these non-idealities during learning tasks. These methods rely on a phenomenological model of the device, obtainable experimentally, and bi-level optimization. We showcase this effect through meta-learning and a differentiable model of conductance updates on few-shot learning tasks. Since pre-training is a necessary procedure for any online learning scenario at the edge, our results may pave the way for real-world applications of memristive devices without significant adaptation overhead.
Furthermore, by considering the programming of memristive devices as a learning problem in its own right, we demonstrate that the developed methods can accelerate existing write-verify techniques.

This work was sponsored by the Federal Ministry of Education, Germany (project NEUROTEC-II grant no. 16ME0398K and 16ME0399) and Neurosys as part of the initiative "Cluster4Future" funded by the Federal Ministery of Education and Research BMBF (03ZU1106CB).

We use our own and third party cookies for analysing and measuring usage of our website to improve our services. If you continue browsing, we consider accepting its use. You can check our Cookies Policy in which you will also find how to configure your web browser for the use of cookies. More info