A universal compression theory for lottery ticket hypothesis and neural scaling laws
Published in arXiv preprint (accepted to ICLR 2026), 2025
This paper gives a constructive compression theory for large models and datasets. It proves that generic permutation-invariant functions of many objects can be compressed to polylogarithmic size with vanishing error, implying that wide neural networks and large datasets can be strongly compressed while preserving learning dynamics and loss landscapes, with direct consequences for lottery-ticket-style compression and neural scaling laws.
Preprint: arXiv:2510.00504
Recommended citation: Hong-Yi Wang, Di Luo, Tomaso Poggio, Isaac L. Chuang, and Liu Ziyin. "A universal compression theory for lottery ticket hypothesis and neural scaling laws." arXiv:2510.00504 (2025). Accepted to ICLR 2026.
Download Paper