search

mclaughlin6464 / pearce

A (better structured) package that implements the calculation of mock cosmological observables and their emulation.
MIT License
0 stars 1 forks source link

Modify training data to have a file dictionary. #4

Closed mclaughlin6464 closed 7 years ago

mclaughlin6464 commented 7 years ago

I had an idea which will dramatically speed up the file IO. I can pickle a dictionary with the key being a (named) tuple of the emulation parameters at a point, and the values being the location of that file. This saves me having to open every file looking for the ones I want. This is something I should consider implementing as the size of datasets grows larger.

mclaughlin6464 commented 7 years ago

This is somewhat urgent to do as loading data and building the emulator are becoming prohibitively slow.

mclaughlin6464 commented 7 years ago

This is completed.