Document Type
Article
Publication Date
8-26-2023
Original Citation
Choudhary A,
Radhakrishnan A,
Lindner J,
Sinha S,
Ditto W.
Neuronal diversity can improve machine learning for physics and beyond. Sci Rep. 2023; 13(1):13962
Keywords
JGM
ISSN
2045-2322
PMID
37634029
DOI
https://doi.org/10.1038/s41598-023-40766-6
Grant
This research was supported by O.N.R. Grant N00014-16-1-3066 and a gift from United Therapeutics. S.S. acknowledges support from the J.C. Bose National Fellowship (Grant No. JBR/2020/000004). W.L.D. thanks Kathleen Russell for the conceptualization of the original idea along with many subsequent discussions.
Abstract
Diversity conveys advantages in nature, yet homogeneous neurons typically comprise the layers of artificial neural networks. Here we construct neural networks from neurons that learn their own activation functions, quickly diversify, and subsequently outperform their homogeneous counterparts on image classification and nonlinear regression tasks. Sub-networks instantiate the neurons, which meta-learn especially efficient sets of nonlinear responses. Examples include conventional neural networks classifying digits and forecasting a van der Pol oscillator and physics-informed Hamiltonian neural networks learning Hénon-Heiles stellar orbits and the swing of a video recorded pendulum clock. Such learned diversity provides examples of dynamical systems selecting diversity over uniformity and elucidates the role of diversity in natural and artificial systems.
Comments
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.