search

Chris-Pedersen / LearnableWavelets

Learnable wavelet neural networks
5 stars 0 forks source link

Simplify loading of maps in dataloader #30

Closed Chris-Pedersen closed 2 years ago

Chris-Pedersen commented 2 years ago

Currently maps are loaded, a subset taken using the splits variable, and a new file created with this subset. This file is then passed to the dataloader. This can create issues with multiple runs simulatenously using different sizes of training sets, adding an extra layer of bookkeeping for these additional "temporary" files. It might make more sense to simplify this and just pass the path to the full sim set straight to the dataloader. This is certainly simpler in the case of a single field, but might be more complicated in the case of multiple fields.

Chris-Pedersen commented 2 years ago

Fixed in d5c5e08 - have generated sets of maps of various sizes in .npy format, and now just pass these paths to the dataloder.