nllb-200-distilled-600M prunes itself to death?

System Info

This is with Transformerrs.js V2

Environment/Platform

[X] Website/web-app
[ ] Browser extension
[ ] Server-side (e.g., Node.js, Deno, Bun)
[ ] Desktop app (e.g., Electron)
[ ] Other (e.g., VSCode extension)

Description

I was testing the nllb-200-distilled-600M model today after running into it in the React translation demo. I was hoping it would generate better translations than Opus MT (So far: not really).

But while testing something odd happened. The test code runs when the model has just been downloaded fresh. But after reloading the page, and grabbing the model from cache, it no longer works.

The model is found. There is once again a huge list of messages that state that all kinds of layers should be pruned form the ONNX model.

But then it just stops.

This is most likely an issue with my code, or the fact that I'm not running it inside a webworker. But I thought I'd share it just in case it resonates with something.

Files:

onnx/encoder_model_quantized.onnx
onnx/decoder_model_merged_quantized.onnx

Reproduction

TEST CODE

import { pipeline, env } from './js/transformers.js'; // V2

self.device = 'wasm';

try{

    console.log("TRANSLATION WORKER: env.backends.onnx.wasm.proxy before: ", env.backends.onnx.wasm.proxy);
    env.backends.onnx.wasm.proxy = self.device !== 'webgpu';
    console.log("TRANSLATION WORKER: env.backends.onnx.wasm.proxy after: ", env.backends.onnx.wasm.proxy);

    env.allowLocalModels = false;
    env.allowRemoteModels = true;
    env.useBrowserCache = true;

    class MyTranslationPipeline {
        static task = 'translation';
        static model = 'Xenova/nllb-200-distilled-600M';
        static instance = null;

        static async getInstance(progress_callback = null) {
            if (this.instance === null) {
                this.instance = pipeline(this.task, this.model, { progress_callback }); // dtype:':'fp16', , 'device':self.device
            }

            return this.instance;
        }
    }

    console.error("CREATING TRANSLATOR PIPELINE");
    let translator = await MyTranslationPipeline.getInstance(x => {
        // We also add a progress callback to the pipeline so that we can
        // track model loading.
        console.log("translator loading: ", x);
        //self.postMessage(x);
    });

    console.error("CALLING TRANSLATOR PIPELINE");

    // Polish
    //let text = 'Marrakesz: To najbardziej znane miasto i stolica Maroka. Jest pełne kolorowych uliczek, meczetów i pysznej kuchni.';

    // Wikipedia
    let text = "The Rottweiler is a breed of domestic dog, regarded as medium-to-large or large. The dogs were known in German as Rottweiler Metzgerhund, meaning Rottweil butchers' dogs, because their main use was to herd livestock and pull carts laden with butchered meat to market. This continued until the mid-19th century when railways replaced droving. Although still used to herd stock in many parts of the world, Rottweilers are now also used as search and rescue dogs, guard dogs, and police dogs.";

    // Hansel and Gretel
    // They had very little to bite or to sup, and once, when there was great dearth in the land, the man could not even gain the daily bread.
    //  As he lay in bed one night thinking of this, and turning and tossing, he sighed heavily, and said to his wife, "What will become of us? we cannot even feed our children; there is nothing left for ourselves.

    let output = await translator(text, {
        src_lang: 'eng_Latn',
        tgt_lang: 'nld_Latn',

        // Allows for partial output
        callback_function: x => {
            console.log("partial translation: ", x);
        }
    });

    console.log("TRANSLATION RAW OUTPUT: ", output);

    let result = translator.tokenizer.decode( output.output_token_ids, { skip_special_tokens: true });

    console.log("FINAL TRANSLATION: ", result);

}
catch(err){
    console.error("caught error in translation test: ", err);
}

xenova / transformers.js