CambridgeCIA / LION

Learned Iterative Optimization Networks
GNU General Public License v3.0
20 stars 15 forks source link

add normalization to LionSolver #117

Closed cshoebridge closed 4 months ago

cshoebridge commented 4 months ago

Zak mentioned that it is often beneficial to normalize inputs. This is done using global dataset statistics, so shouldn't be the job of a LIONModel, who has no knowledge of the dataset, rather the job of the LIONSolver who does.

Added functions to optionally enable normalization and logic in existing solvers to leverage this.

AnderBiguri commented 4 months ago

Great idea! I wonder however if we should pre-do this. Iterating over the entire dataset can be super long, so maybe this should be in each DataSet rather than in LIONSolver?

Unsure, actually, so maybe a thing worth discussing.

AnderBiguri commented 4 months ago

On second thought, even if it was in the datasets we provide, it would also need to be in the solver to be able to support generic datasets, so it makes sense

AnderBiguri commented 4 months ago

This has been tested I assume? happy to merge?

cshoebridge commented 4 months ago

yes good idea, could be worth looking at, though the inner working of the datasets is out of my realm of expertise

cshoebridge commented 4 months ago

and yes, tested, as set up it doesn't break anything else, it's entirely optional.