huggingface / transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!
https://huggingface.co/docs/transformers.js
Apache License 2.0
11.36k stars 708 forks source link

Trying to load a model with use_external_data_format in V3 alpha fails #963

Open benc-uk opened 1 week ago

benc-uk commented 1 week ago

System Info

Version: transformers@3.0.0-alpha.20 Runtime: Vanilla JS (ESM) in browser, no React or Next.js

Environment/Platform

Description

Trying to create a pipeline with the Phi-3 models microsoft/Phi-3-mini-4k-instruct-onnx-web or Xenova/Phi-3-mini-4k-instruct which both have external data files, results in a error message that the external data file is not available (which is incorrect, the file exists!),
I am setting use_external_data_format: true and dtype: "q4f16" (or q4 in the case of the Xenova version of the model)

I don't believe this is a dupe of this issue, as I'm not using Node.js

image

Error: Can't create a session. ERROR_CODE: 1, 
ERROR_MESSAGE: Deserialize tensor model.layers.27.mlp.gate_proj.MatMul.weight_Q4 failed.Failed to load external data file ""model_q4f16.onnx_data"", error: Module.MountedFiles is not available.

Reproduction

Try to create a pipeline with use_external_data_format: true e.g.

import { env, pipeline } from 'https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.0.0-alpha.19'

pipeline = await pipeline('text-generation', 'microsoft/Phi-3-mini-4k-instruct-onnx-web', {
  device: 'webgpu',
  use_external_data_format: true,
  dtype: 'q4f16',
})