Storage of Downloaded Models from HuggingFace

chengchingwen / Transformers.jl

Julia Implementation of Transformer models

MIT License

526 stars 75 forks source link

Storage of Downloaded Models from HuggingFace #177

Open kadir-gunel opened 6 months ago

kadir-gunel commented 6 months ago

Hello,

I was wondering if there is any special reason that the downloaded models are stored in cache folders? Since models are downloaded from HuggingFace wouldn't be nice to save each downloaded model under the /home/user_name/.cache/huggingface/hub?

Best Regards

chengchingwen commented 6 months ago

There were a few reasons that we did not share the cache with huggingface/hub (but I'm not sure if they still hold now). To put it simply, I want to 1. avoid storing duplicate files 2. make sure the files are read-only, and 3. ease the development pressure of tracking the changes of their cache format/specification.