search

huggingface / transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!
https://huggingface.co/docs/transformers.js
Apache License 2.0
12.22k stars 776 forks source link

Is it possible to fine-tune the hosted pretrained models? #522

Open lhohoz opened 10 months ago

lhohoz commented 10 months ago

Question

Hello, If we have a large dataset in our domain, can we use it to fine-tune the hosted pretrained models(for example: Xenova/nllb-200-distilled-600M) with optimum? or is it possible to convert our own translation Pytorch model to ONNX which can be compatible with transformer.js?

DavidGOrtega commented 10 months ago

👋 there is a tutorial on how you have to compile your model to onnx. Basically you should train your model with pytorch and then convert the model to onnx. That would suffice as transformers.js mimics transformers using same tokenizers, etc...

xenova commented 10 months ago

Hi there @lhohoz 👋 As stated in the model card of Xenova/nllb-200-distilled-600M, it is a fork of https://huggingface.co/facebook/nllb-200-distilled-600M (original model), just with ONNX weights to be compatible with transformers.js. For that reason, you can finetune the original model with the python transformers library, and then afterwards, convert it to ONNX to be run in transformers.js 😇

lhohoz commented 10 months ago

Thanks all, will try