Joshuaalbert / DSA2000-Cal

DSA-2000 Calibration and Forward Modelling
https://www.deepsynoptic.org/overview
MIT License
1 stars 1 forks source link

Model compression + chunked evaluation #73

Open Joshuaalbert opened 4 months ago

Joshuaalbert commented 4 months ago

We can compress models in the following way:

This means we should enforce this at the model construction level, thus using DI we should abstract the predict to be derivable from the model level. Is this a big rewrite?

For chunked evaluation, we first point out that data can already be sharded over devices according to frequency. Local shards are thus meant to be handled by the local device. Then chunked processing should use scan(vmap) to processing chunks sequentially. According to #72 we know this is better than vmap(scan).