Faculty of Science University of Ontario Institute of Technology 2000 Simcoe St. N., Oshawa ON L1G 0C5
This work studies the generality of layers across a continuously-parametrized set of tasks: a group of similar problems whose details are changed by varying a real number. We found the transfer learning method for measuring generality prohibitively expensive for this task. Instead, by relating generality to similarity, we develop a computationally efficient measure of generality that uses the singular vector canonical correlation analysis (SVCCA). We demonstrate our method by measuring layer generality in neural networks trained to solve differential equations.
Layer generality in DENNs using SVCCA
Intrinsic dimensionality, reproducibility, and specificity