search

bespokelabsai / curator

Apache License 2.0
14 stars 1 forks source link

inspect(func) is sensitive to comments and whitespace - cache invalidates #124

Open RyanMarten opened 5 hours ago

RyanMarten commented 5 hours ago

@vutrung96

just black . your code invalidates the cache

RyanMarten commented 4 hours ago

Related to #127

RyanMarten commented 4 hours ago

https://huggingface.co/docs/datasets/en/about_cache

Huggingface uses dill and pickle

Source code for their hasher: https://huggingface.co/docs/datasets/v3.1.0/en/package_reference/main_classes#datasets.fingerprint.Hasher

@vutrung96 @CharlieJCJ