search

jncraton / languagemodels

Explore large language models in 512MB of RAM
https://jncraton.github.io/languagemodels/
MIT License
1.18k stars 78 forks source link

How to make it run offline? #24

Closed Luckyoriginal closed 1 year ago

Luckyoriginal commented 1 year ago

when i use it and then turn off the internet, it still works. but when i turn off the internet before launching it, it doesn't work anymore. is there any way to make it work offline?

jncraton commented 1 year ago

This works for me on my system. I can run scripts offline as long as the models are already downloaded. Could you share a little more information about your setup so that I can reproduce this?

Luckyoriginal commented 1 year ago

Sorry, it worked but it's slower starting offline so I thought it wasn't working. Is there a reason for that?

jncraton commented 1 year ago

Without knowing more about your environment, it is difficult to say for sure, but my guess is that your system is waiting on a timeout when checking for model updates. You should be able to skip those checks by setting the HF_HUB_OFFLINE environment variable:

HF_HUB_OFFLINE

If set, no HTTP calls will me made when trying to fetch files. Only files that are already cached will be accessed. This is useful in case your network is slow and you don’t care about having absolutely the latest version of a file.

Note: even if the latest version of a file is cached, calling hf_hub_download still triggers a HTTP request to check that a new version is not available. Setting HF_HUB_OFFLINE=1 will skip this call which speeds up your loading time.

Let me know if that helps.

Luckyoriginal commented 1 year ago

it work ! thank you