Make storing single-threaded to fix hugginface commit issues

openclimatefix / nwp-consumer

Microservice for consuming NWP data.

9 stars 3 forks source link

Make storing single-threaded to fix hugginface commit issues #93

Closed devsjc closed 8 months ago

devsjc commented 8 months ago

Huggingface does not like simultaneous file pushes as it is a version-control based system that checks the head commit hasn't changed before every push. Since this process is not time-critical, revert it to be done in a single-threaded manner.