valeriupredoi
commented
2 years ago this is a bit Chinese for me atm - can we talk with voice about it maybe? :cn:
Open ledm opened 2 years ago
valeriupredoi
commented
2 years ago this is a bit Chinese for me atm - can we talk with voice about it maybe? :cn:
The Analysis_timeseries.py shelves files don't follow best practice. As it stands, they currently fully re-written regularly. It would be better to use the shelve sync function to update on disk.
It's not a huge issue as there is not a lot of data to write most of the time, but it could become more significant on days where jasmin is slow, or when we're looking ar very large shelve files (hundreds or thousands of years).
It would also be nice to hear your thoughts on shelve files in general, @valeriupredoi. They're so easy to use, but have some risks. We could reduce some of their volatility by for instance requesting a specific pickle protocol to reduce uncertainty.