Using api object, can access server, send commands, receive stats in a separate script
Should ease integrating CHEESE into other projects
Checkpointing fix: [WIP]
Due to an issue with huggingface datasets, it is non-trivial to add more data to a dataset that has already been saved to disk
Creating a new dataset -> saving to disk -> later loading from disk -> adding new elements -> saving to disk again, results in error. Currently reaching out to HF community to find a solution.
In general, it seems non-trivial to overwrite a dataset if it has already been saved in an independent script
API Upgrade:
Checkpointing fix: [WIP]