Transformers.js seems to need an internet connection when it shouldn't? (Error: no available backend found.)

[flatsiedatsie]

Question

What is the recommended way to get Transformers.js to work even when, later on, there is no internet connection?

Is it using a service worker? Or are there other (perhaps hidden) settings for managing caching of files?

I'm assuming here that the Error: no available backend found error message is related to Transformers.js not being able to find files once Wi-Fi has been turned off. I was a bit surprised by that, since I do see a cache called transformers-cache being created. Is that not caching all the required files?

[flatsiedatsie]

Looking a bit further at which files are in transformers-cache (for a translation pipeline):

"https://huggingface.co/Xenova/opus-mt-nl-en/resolve/main/tokenizer_config.json": "transformers-cache",
    "https://huggingface.co/Xenova/opus-mt-nl-en/resolve/main/config.json": "transformers-cache",
    "https://huggingface.co/Xenova/opus-mt-nl-en/resolve/main/tokenizer.json": "transformers-cache",
    "https://huggingface.co/Xenova/opus-mt-nl-en/resolve/main/generation_config.json": "transformers-cache",
    "https://huggingface.co/Xenova/opus-mt-nl-en/resolve/main/onnx/encoder_model_quantized.onnx": "transformers-cache",
    "https://huggingface.co/Xenova/opus-mt-nl-en/resolve/main/onnx/decoder_model_merged_quantized.onnx": "transformers-cache",
    "https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/tokenizer_config.json": "transformers-cache",
    "https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/preprocessor_config.json": "transformers-cache",
    "https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/config.json": "transformers-cache",
    "https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/generation_config.json": "transformers-cache",
    "https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/tokenizer.json": "transformers-cache",
    "https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/onnx/encoder_model_quantized.onnx": "transformers-cache",
    "https://huggingface.co/Xenova/whisper-tiny.en/resolve/main/onnx/decoder_model_merged_quantized.onnx": "transformers-cache",

The transformers.jsfile itself doesn't seem to be in that cache, only models.

I'm assuming that's by design.

[jonathanpv]

Hmm, I'm fairly new to webdev but this information may help?

[jonathanpv]

If you find anything out please share :) I'd like my app to be working offline too

[flatsiedatsie]

I actually did go that serviceworker route. Have a look at the long script at the bottom here.

But what I find strange is this situation:

• I'm running a development version of my project on localhost.
• transformers.js and it's files are in a local subfolder, e.g. localhost.dd/project/js
• If I turn off wifi, I would expect the localy hosted version of Transformers.js to work, since it's... available. The model files it needs are also available, since it does cache those.

But instead I get this error. So Transformers.js - or something it relies on - seems to need an internet connection when it shouldn't?

[flatsiedatsie]

OMG, of course. It must be the env settings.

env.allowLocalModels = true;
//env.allowRemoteModels = false;

[Th3G33k]

@flatsiedatsie

You should try setting those env settings :

env.backends.onnx.wasm.wasmPaths
env.localModelPath 
env.useBrowserCache
env.allowRemoteModels

By default, the wasm file is downloaded from cdn.

If you are using cache, make sure that there is no error404 page cached. Otherwise, for json and onnx, it will not redirect to huggingface.co, and will always retrieve the error pages.

[flatsiedatsie]

@Th3G33k Thanks! That's very helpful. It might help me squash this rimple:

[flatsiedatsie]

I'll close this for now actually, since the original question is answered:

• Transformers.js does cache models it downloads, but it doesn't cache itself. You have to implement that, either by some cache.add() functionality, or by implementing a service worker.
• The question about "it seems to need internet" was user error on my part, as I didn't understand what all the env options did, and in the code I copied 'from teh internet' there was a setting that disabled grabbing files locally. Doh.

[MarketingPip]

@flatsiedatsie - tho this issue is closed and I do not need it etc.... I am VERY IMPRESSED with your last comment to help other users / developers when facing issues like this. (Hate when people do not summarize things like this for other developer's when user prone issue). Keep up the good work.