Open jason-trinidad opened 10 months ago
xenova
commented
10 months ago Could you try first convert the model to be compatible with pytorch before running the conversion script? Admittedly, I haven't tried converting directly to ONNX from a tensorflow model.
jason-trinidad
commented
10 months ago Got a chance to try this - I re-trained the model using PyTorch and re-ran the script. Looks like the ONNX export succeeded this time! Thanks for the tip.
I did run into an error with quantization:
"""
Quantizing: 0%| | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
File "
And another error when I tried to load my model from Huggingface using transformers.js'.
Via AutoModel.from_pretrained('[HF location]', {'quantized': false}):
""" Error: Could not locate file: "https://huggingface.co/jason-trinidad/t5-small-ft-120123-torch/resolve/main/onnx/decoder_model_merged.onnx". at handleError (hub.js:240:11) at getModelFile (hub.js:473:24) at async constructSession (models.js:119:18) at async Promise.all (:5173/index 2) at async T5Model.from_pretrained (models.js:750:20) at async AutoModel.from_pretrained (models.js:3960:20) at async loadModel (List.jsx:41:21) """
Quantization is not crucial for me at the moment. But I still get the issue if I try the script without the quantization flag. Do you know how I resolve this issue with the merged file?
xenova
commented
10 months ago This looks like a dependency version issue. Can you try with this list of requirements and our conversion script? The combination should work for t5 models.
jason-trinidad
commented
10 months ago Hi sorry for the delay. Tried this on a new env with the requirements you linked to, I got the same error. It's looking for decoder_model_merged.onnx. Any ideas?
xenova
commented
9 months ago If you could provide the ID of the model you are looking to convert, I could try it on my side. Are you converting the model with optimum directly or with our conversion script?
jason-trinidad
commented
9 months ago Thanks, I uploaded it to "jason-trinidad/t5-ft-to-convert" on HF. I was using your conversion script.
xenova
commented
9 months ago Thanks! I did the conversion myself and I didn't run into any issues. I opened a PR to your repo and ran it with the following code:
import { pipeline } from '@xenova/transformers';
const pipe = await pipeline('text2text-generation', 'jason-trinidad/t5-ft-to-convert', {
// quantized: false, Uncomment this line to use the unquantized version
revision: 'refs%2Fpr%2F1',
});
const result = await pipe('Translate to German: Hello', {
max_new_tokens: 50,
});
console.log(result);
// [{ generated_text: 'Hallo' }]
shizheng-rlfresh
commented
8 months ago Hi! I had similar issue and I used the conversion script + requirement dependency. In addition, I always got an error of "unsupported model IR version: 9, max supported IR version 8. My ID is shi-zheng-qxhs/gpt2_oasst2_curated
SmarMykel
commented
5 months ago @shizheng-rlfresh Were you able to fix the "unsupported model IR version: 9, max supported IR version 8"
shizheng-rlfresh
commented
5 months ago @SmarMykel have you tried the recommended script? try use a venv and run the script...see if that works 😄
python -m venv .venv
# activate .venv and install required packages
source .venv/bin/activate
pip install -r requirements.txt
# run the conversion script - <modelid>
python -m scripts.convert --quantize --model_id <modelid>Its working thank you!!! @shizheng-rlfresh
Describe the bug A clear and concise description of what the bug is.
On my M1 Mac, the convert script throws an unexpected error when I try to use it on my model.
How to reproduce Steps or a minimal working example to reproduce the behavior
I fine-tuned an instance of
t5-small, then saved the model and tokenizer to a directory via[model/tokenizer].save_pretrained(). (The model saved as a.h5file.) I then ran:python -m [copy of convert script] --quantize --task 'text2text-generation' --model_id [local directory containing saved files]Expected behavior A clear and concise description of what you expected to happen.
I expected to create a directory with a corrected config file and quantized Transformers.js-compatible model.
Logs/screenshots If applicable, add logs/screenshots to help explain your problem.
Here is the full error and traceback:
Environment