Journal article 1 view
Stochastic weight matrix dynamics during learning and Dyson Brownian motion
Gert Aarts 
,
Biagio Lucini ,
Chanju Park
,
Biagio Lucini ,
Chanju Park
Physical Review E, Volume: 111, Issue: 1
Swansea University Authors:
Gert Aarts , Biagio Lucini
Full text not available from this repository: check for access using links below.
DOI (Published version): 10.1103/physreve.111.015303
Abstract
We demonstrate that the update of weight matrices in learning algorithms can be described in the framework of Dyson Brownian motion, thereby inheriting many features of random matrix theory. We relate the level of stochasticity to the ratio of the learning rate and the minibatch size, providing more...
| Published in: | Physical Review E |
|---|---|
| ISSN: | 2470-0045 2470-0053 |
| Published: |
American Physical Society (APS)
2025
|
| Online Access: |
Check full text
|
| URI: | https://cronfa.swan.ac.uk/Record/cronfa68607 |
| Abstract: |
We demonstrate that the update of weight matrices in learning algorithms can be described in the framework of Dyson Brownian motion, thereby inheriting many features of random matrix theory. We relate the level of stochasticity to the ratio of the learning rate and the minibatch size, providing more robust evidence to a previously conjectured scaling relationship. We discuss universal and nonuniversal features in the resulting Coulomb gas distribution and identify the Wigner surmise and Wigner semicircle explicitly in a teacher-student model and in the (near-)solvable case of the Gaussian restricted Boltzmann machine. |
|---|---|
| College: |
Faculty of Science and Engineering |
| Funders: |
G.A. andB.L. are supported by STFC Consolidated Grant No. ST/X000648/1 .B. L. is further supported by the UKRI EPSRC ExCALIBURExaTEPP Project No. EP/X017168/1. C. P. is supported by the UKRI AIMLAC CDT EP/S023992/1. |
| Issue: |
1 |