search

winstxnhdw / nllb-api

A performant high-throughput CPU-based API for Meta's No Language Left Behind (NLLB) using CTranslate2, hosted on Hugging Face Spaces.
https://huggingface.co/spaces/winstxnhdw/nllb-api
96 stars 18 forks source link

start container offline #132

Closed Q-si closed 8 months ago

Q-si commented 8 months ago

Hello, I can't start the docker container successfully when I try to use the system completely offline. I don't get any error, but the swagger page is not reachable. As soon as I do an activation on huggingface.co, it works. What is the reason for this and can I work around it?

winstxnhdw commented 8 months ago

You have to download and cache the model before trying to run it offline.

See this: https://github.com/winstxnhdw/nllb-api?tab=readme-ov-file#model-caching

Q-si commented 7 months ago

Thank you for your response. But i cached it locally on the host (-v). When I unplug the network and try to start the container, I get the error that no connection to huggingface.co can be established. I have attached a screenshot for the online/offline run-command. Screenshot_offline Screenshot_online

winstxnhdw commented 7 months ago

Okay, I've pushed a fix for this in the latest commit.

Q-si commented 7 months ago

It works! You are my hero :)