A Brief Overview of the Typology of Neurons and Analysis of Using Memristor Crossbars

Cover Page

Cite item

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription Access

Abstract

Neuromorphic technologies using artificial neurons and synapses can offer a more efficient solution for the execution of artificial intelligence algorithms than traditional computing systems. Artificial neurons using memristors have recently been developed, but they have limited biological dynamics and cannot interact directly with artificial synapses in an integrated system. The purpose of the work is to review the levels of complexity and functions of neurons and synapses, as well as to analyze the circuitry of certain types of neurons and neural networks.

Full Text

Restricted Access

About the authors

A. A. Tokarev

MIREA — Russian Technological University (RTU MIREA)

Author for correspondence.
Email: santokar5@gmail.com
Russian Federation, Moscow

I. A. Khorin

K.A. Valiev Institute of Physics and Technology of the Russian Academy of Sciences

Email: khorin@ftian.ru
Russian Federation, Moscow

References

  1. Strukov D.B., Snider G.S., Stewart D.R., Williams R.S. The missing memristor found // Nature, 2008. V. 453. P. 80–83.
  2. Roy K., Laiswal A., Panda P. Towards spike-based machine intelligence with neuromorphic computing // Nature, 2019. V. 575. P. 607—617.
  3. Jeon W.K.G., Lee J., Lee H. et al. Deep learning with GPUs. In S. Kim, & G. C. Deka (Eds.), Hardware Accelerator Systems for Artificial Intelligence and Machine Learning (2021). (pp. 167—215). (Advances in Computers; Vol. 122). Academic Press Inc.
  4. Capra M., Bussolino B., Marchisio A. et al. Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead // IEEE Access, 2020. V. 8. P. 225134—225180.
  5. Sebastian A., Le Gallo M., Khaddam-Aljameh R. et al. Memory devices and applications for in-memory computing // Nat. Nanotechnol., 2020. V. 15. P. 529—544.
  6. Xia Q.-F., Yang J.-J. Memristive crossbar arrays for brain-inspired computing // Nat. Mater., 2019. V. 18(4). P. 309–323.
  7. Jo S.H., Chang T., Ebong I. et al. Nanoscale memristor device as synapse in neuromorphic systems // Nano Lett., 2010. V. 10. P. 1297–1301.
  8. Yu S., Wu Y., Jeyasingh R. et al. An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation // IEEE Trans. Electron Devices, 2011, V. 58. P. 2729–2737.
  9. Ohno T., Hasegawa T., Tsuruoka T. et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses // Nat. Mater., 2011. V. 10. P. 591–595.
  10. Pershin Y.V., Di Ventra M. Neuromorphic, digital, and quantum computation with memory circuit elements // Proc. IEEE, 2012. V. 100. P. 2071–2080.
  11. Hu S.G., Liu Y., Liu Z. et al. Associative memory realized by a reconfgurable memristive Hopfeld neural network // Nat. Commun., 2015. V. 6. 7522 (2015).
  12. Burr G.W., Shelby R.M., Sidler S. et al. Experimental demonstration and tolerancing of a large-scale neural network (165 000 synapses) using phase-change memory as the synaptic weight element // 2014 IEEE International Electron Devices Meeting, pp. 29.5.1—29.5.4.
  13. Wong H.−S.P., Lee H.−Y., Yu S. et al. Metal−oxide RRAM // Proceedings of the IEEE, 2012. V. 100. № 6. P. 1951—1970.
  14. Yang J.J., Strukov D.B., Stewart D.R. Memristive devices for computing // Nature Nanotechnology, 2013. V. 8, № 1. P. 13—24.
  15. van de Burgt Y., Lubberman E., Fuller E.J. et al. A non-volatile organic electrochemical device as a low-voltage artifcial synapse for neuromorphic computing // Nat. Mater., 2017. V. 16(4). P. 414–418.
  16. Merolla P.A., Arthur J.V., Alvarez-Icaza R. et al. A million spiking-neuron integrated circuit with a scalable communication network and interface // Science, 2014. V. 345. P. 668–673.
  17. Sourikopoulos I., Hedayat S., Loyez C. et al. A 4-fJ/spike artifcial neuron in 65 nm CMOS technology // Front. Neurosci., 2017. V. 11. 123.
  18. Li C., Belkin D., Li Y. et al. Efficient and self-adaptive in-situ learning in multilayer memristor neural networks // Nat. Commun., 2018. V. 9. 2385.
  19. Trepel M. Neuranatomie: Struktur und Funktion 7. Auflage, Elsevier. Munchen. 2012. 439 P.
  20. Mead C. Neuromorphic electronic systems // Proc. IEEE, 1990. V. 78. P. 1629–1636.
  21. Kheradpisheh S.R., Ganjtabesh M., Thorpe S.J. et al. STDP-based spiking deep convolutional neural networks for object recognition // Neural Networks, 2018. V. 99. P. 56—67.
  22. Mozafari M., Ganjtabesh M., Nowzari-Daliniet A. al. Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks // Pattern Recognition, 2019. V. 94. P. 87—95.
  23. Peng H., Gan L., Guo X. Memristor based Spiking Neural Networks: Cooperative Development of Neural Network Architecture/Algorithms and Memristors // Chip, 2024. V. 3(2). 100093.
  24. Ianov D., Chezhegov A., Kiselev M. et al. Neuromorphic artificial intelligence systems // Frontiers in Neuroscience, 2022. doi: 10.3389/fnins.2022.959626
  25. Chen A., Datta S., Hu X.S. et al. A survey on architecture advances enabled by emerging beyond-CMOS technologies // IEEE Design & Test, 2019. V. 36. P. 46–68.
  26. Dragoman M., Dragoman D. Atomic-Scale Electronics Beyond CMOS. Springer Nature Switzerland AG. 2021.
  27. Hebb D.O. The Organization of Behavior: A Neuropsychological Theory. John Wiley & Sons, Inc. 1949.
  28. Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain // Psychol. Rev., 1958, V. 65. 386.
  29. Ивахненко А.Г., Лапа В.Г. Кибернетические предсказывающие устройства. Киев – 1965. 219 с.
  30. Werbos P.J. Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. Harvard Univ. Cambridge, Massachusetts. 1974. 453 p.
  31. Kelley H.J. Gradient theory of optimal flight paths // ARS J., 1960. V. 30. P. 947–954.
  32. Rumelhart D.E., Hinton G.E., Williams R.J. Learning representations by back-propagating errors // Nature, 1986. V. 323. P. 533–536.
  33. Hopfield J.J. Neural networks and physical systems with emergent collective computational abilities // Proc. Natl Acad. Sci. USA, 1982. V. 79. P. 2554–2558.
  34. Kohonen T. Self-organized formation of topologically correct feature maps // Biol. Cybern., 1982. V. 43. P. 59–69.
  35. Manipatrun, S., Nikonov D.E., Young I.A. Beyond CMOS computing with spin and polarization // Nat. Phys., 2018. V. 14. P. 338–343.
  36. Preskill J. Quantum computing in the NISQ era and beyond // Quantum, 2018. V. 2. P. 79.
  37. Solli D.R., Jalali B. Analog optical computing // Nat. Photonics, 2015. V. 9. P. 704–706.
  38. Паун Г., Розенберг Г., Саломаа А. ДНК-компьютер. Новая парадигма вычислений: пер. с англ. – М.: Мир, 2003. – 528 с.
  39. Mead C. Neuromorphic electronic systems // Proc. IEEE. 1990. V. 78. P. 1629–1636.
  40. Kephart J.O., Chess D.M. The vision of autonomic computing // Computer, 2003. V. 36. P. 41–50.
  41. Kish L.B. Thermal noise driven computing // Appl. Phys. Lett., 2006. V. 89. 144104.
  42. Unger R., Moult J. Towards computing with proteins // Proteins, 2006. V. 63. P. 53–64.
  43. Upadhyay N.K., Jiang H., Wang Z. et al. Emerging memory devices for neuromorphic computing // Adv. Mater. Technol., 2019. V. 4. 1800589.
  44. Bengio Y. Deep Learning of Representations for Unsupervised and Transfer Learning // JMLR: Workshop and Conference Proceedings, 2012, V. 27. P. 17—37.
  45. Schmidhuber J. Deep learning in neural networks: An overview // Neural Netw., 2015. V. 61 P. 85–117.
  46. Chua L.O. Local activity is the origin of complexity // Int. J. Bifurc. Chaos, 2005. V. 15. P. 3435–3456.
  47. Csete M.E., Doyle J.C. Reverse engineering of biological complexity // Science, 2002. V. 295. P. 1664–1669.
  48. Stanley H.E., Amaral L.A., Buldyrev S.V. et al. Self-organized complexity in economics and finance // Proc. Natl Acad. Sci. USA, 2002. V. 99. P. 2561–2565.
  49. Arthur W.B. Foundations of complexity economics // Nat. Rev. Phys., 2021. V. 3. P. 136–145.
  50. Wilson H.R. Voluntary generation of hyperchaotic visuo-motor patterns // Sci. Rep., 2019. V. 9. 13819.
  51. Yao P., Wu H., Gao B., et al. Fully hardware-implemented memristor convolutional neural network // Nature, 2020. V. 577 P. 641—646.
  52. Hodgkin A.L., Huxley A.F. A quantitative description of membrane current and its application to conduction and excitation in nerve // J. Physiol., 1952. V. 117(4). P. 500–544.
  53. Kistler W.G.W.M. Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press. 2002. 498 p.
  54. Izhikevich E.M. Simple model of spiking neurons // IEEE Trans. Neural Netw., 2003. V. 14. P. 1569–1572.
  55. Segee B. Methods in Neuronal Modeling: from Ions to Networks, 2nd Edition // Comput. Sci. Eng., 1999. V. 1(1). P. 81—81.
  56. Taherkhani A., Belatreche A., Liet Y. al. A review of learning in biologically plausible spiking neural networks // Neural Netw., 2020. V. 122. P. 253—272.
  57. Fang W., Yu Z., Chen Y. et al. Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks // 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, QC, Canada, 2021. P. 2641—2651.
  58. Duan Q., Jing Z., Zou X. et al. Spiking neurons with spatiotemporal dynamics and gain modulation for monolithically integrated memristive neural networks // Nat. Commun., 2020. V. 11. 3399.
  59. Wang Y.-H., Gong T.-C., Ding Y.-X. et al. Redox memristors with volatile threshold switching behavior for neuromorphic computing // Journal of Electronic Science and Technology, 2022. V. 20. 100177.
  60. Wang Z.-R., Joshi S., Savel’ev S. et al. Fully memristive neural networks for pattern classification with unsupervised learning // Nat. Electron., 2018. V. 1(2). P. 137–145.
  61. Kim Y., Kwon Y.J., Kwon D.E., Yoon K.J. Nociceptive Memristor // Adv. Mater., 2018. V. 30, 1704320.
  62. Davies M., Srinivasa N., Lin T.-H., et al. Loihi: a neuromorphic manycore processor with on-chip learning // IEEE Micro, 2018. V. 38. №1. P. 82—99.
  63. Imam N., Cleland T.A. Rapid online learning and robust recall in a neuromorphic olfactory circuit // Nat. Mach. Intell., 2020. V. 2(3). P. 181–191.
  64. Merolla P.A., Arthur J.V., Alvarez-Icaza R., et al. A million spiking-neuron integrated circuit with a scalable communication network and interface // Science, 2014. V. 345(6197). P. 668–673.
  65. DeBole M.V., Taba B., Amir A., et al. TrueNorth: Accelerating from zero to 64 million neurons in 10 years // Computer, 2019. V. 52(5). P. 20—29.
  66. Benjamin B.V., Gao P.-R., Mcquinn E. et al. Neurogrid: a mixed-analog-digital multichip system for large-scale neural simulations // Proc. IEEE, 2014. V. 102(5). P. 699–716.
  67. Beck M.E., Shylendra A., Sangwan V.K., et al. Spiking neurons from tunable Gaussian heterojunction transistors // Nat. Commun., 2020. V. 11(1). P. 1565, 1—8.
  68. Covi E., Donati E., Liang X.-P., et al. Adaptive extreme edge computing for wearable devices // Front. Neurosci., 2021. V. 15. P. 611300, 1—27.
  69. Pickett M.D., Medeiros-Ribeiro G., Williams R.S. A scalable neuristor built with Mott memristors // Nat. Mater., 2013. V. 12(2). P. 114–117.
  70. Yi W., Tsang K.K., Lam S.K., et al. Biological plausibility and stochasticity in scalable VO2 active memristor neurons // Nat. Commun., 2018. V. 9 P. 4661, 1—10.
  71. Gao L.-G., Chen P.-Y., Yu S.-M. NbOx based oscillation neuron for neuromorphic computing // Appl. Phys. Lett., 2017. V. 111(10). P. 103503, 1—4.
  72. Kumar S., Williams R.S., Wang Z.-W. Third-order nanocircuit elements for neuromorphic engineering // Nature, 2020. V. 585. 7826. P. 518–523.
  73. Zhang X.-M., Wang W., Liu Q., et al. An artificial neuron based on a threshold switching memristor // IEEE Electron. Device Lett., 2018. V. 39(2). P. 308–311.
  74. Zhang X.-M., Lu J., Wang Z.-R., et al. Hybrid memristor-CMOS neurons for in-situ learning in fully hardware memristive spiking neural networks // Sci. Bull., 2021. V. 66(16). P. 1624–1633.
  75. Fu T.-D., Liu X.-M., Gao H.-Y., et al. Bioinspired bio-voltage memristors // Nat. Commun., 2020. V. 11(1). P. 1861, 1—10.
  76. Li X.-Y., Tang J.-S., Zhang Q.-T., et al. Power-efficient neural network with artificial dendrites // Nat. Nanotechnol., 2020. V. 15(9) P. 776–782.
  77. Wu W., Wu H., Gao B. et. al. A methodology to improve linearity of analog RRAM for neuromorphic computing // 2018 IEEE Symposium on VLSI Technology, Honolulu, HI, USA, 2018. P. 103–104.
  78. Kim T., Hu S., Kim J. et al. Spiking Neural Network (SNN) With Memristor Synapses Having Non-linear Weight Update // Front. Comput. Neurosci., 2021. V. 15. 646125.
  79. Rao M., Tang H., Wu J. et al. Thousands of conductance levels in memristors integrated on CMOS // Nature, 2023. V. 615. P. 823—829.
  80. Liang S., Yin S., Liu L. et al. FP-BNN: Binarized neural network on FPGA // Neurocomputing, 2018. V. 275. P. 1072—1086.
  81. Simons T., Lee D.-J. A Review of Binarized Neural Networks // Electronics, 2019. V. 8. P. 661.
  82. Zhang W., Yao P., Gao B. et al. Edge learning using a fully integrated neuro-inspired memristor chip // Science, 2023. V. 381. P. 1205—1211.
  83. Balaji A., Das A., Wu Y. et al. Mapping Spiking Neural Networks to Neuromorphic Hardware // IEEE Trans. Very Large Scale Integr. (VLSI) Syst., 2020. V. 28. P. 76—86.
  84. Yadav D.N., Thangkhiew P.L., Chakraborty S., et al. Efficient grouping approach for fault tolerant weight mapping in memristive crossbar array // Memories —Materials, Devices, Circuits and Systems, 2023. V. 4. 100045.
  85. Zhang B., Uysal N., Fan D. et al. Handling Stuck-at-Fault Defects Using Matrix Transformation for Robust Inference of DNNs // IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020. V. 39. №10. P. 2448—2460.
  86. Xu Y., Jin S., Wang Y., Qi Y. Aggressive Fault Tolerance for Memristor Crossbar-Based Neural Network Accelerators by Operational Unit Level Weight Mapping // IEEE Access, 2021. V. 9. P. 102828—102834.
  87. Xia L., Huangfu W., Tang T., et al. Stuck-at Fault Tolerance in RRAM Computing Systems // IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2018. V. 8 №1. P. 102—115.
  88. Li J., Ma L., Sham C.-W., Fu C. A Novel Computing Paradigm for MobileNetV3 using Memristor // arXiv:2402.10512v1 [cs.AR]. 16 Feb 2024. 1—15.
  89. Yan B.-N., Chen Y.-R., Li H. Challenges of memristor based neuromorphic computing system // Sci. China Inf. Sci., 2018. V. 61(6). P. 060425, 1—3.
  90. Sung C., Hwang H., Yoo I.K. Perspective: a review on memristive hardware for neuromorphic computation // J. Appl. Phys., 2018. V. 124(15). P. 151903, 1—13.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download
2. Fig. 1. Examples of synaptic behavior requiring different orders of complexity.

Download (233KB)
Indexing metadata
3. Fig. 2. Examples of neural behavior requiring different orders of complexity.

Download (222KB)
Indexing metadata
4. Fig. 3. Schematic diagrams of (a) the Hodgkin-Huxley neural circuit model and (b) the LIF neural circuit model [59].

Download (117KB)
Indexing metadata
5. Fig. 4. Schematic of the neuristor constructed using two nanoscale memristors according to the Hodgkin-Huxley model [69].

Download (102KB)
Indexing metadata
6. Fig. 5. Schematic illustration of neuron circuitry [76].

Download (74KB)
Indexing metadata
7. Fig. 6. Universal computational architecture of neural networks based on memristors [51].

Download (444KB)
Indexing metadata

Copyright (c) 2024 Russian Academy of Sciences