search

praeclarum / transformers-js

Browser-compatible JS library for running language models
MIT License
217 stars 17 forks source link

What would it take to allow further training of pretrained models in the browser? #2

Open ai-nonbeliever opened 2 years ago

ai-nonbeliever commented 2 years ago

First of all, let me start by complimenting your amazing work.

I am trying to learn AI/ML for the job market. Transformer models seem to be the state-of-the-art AI models, and your work just made it more accessible to everyone.

This is not a bug report, rather it is a question.

I think that it will be great if these pretrained models could be further trained using the browser interface as well. I am wondering whether an additional few series of training data, perhaps a series of questions and answers (may be like initially some text around 10kb in size) can be used to further train a model with some reasonable amount of time and compute power, in the browser itself.

Do you think it is possible, and if yes then do you have a plan to make this feature and user interface, a part of this project?

praeclarum commented 1 year ago

It's possible, yes. The ONNX runtime (which I use) does not have good training support, but tensorflow-js does. I haven't tried training a language model in the browser. I can guarantee it will be slow. :-) Would be interesting to see how long it takes to get accuracy gains from that fine-tuning training.

bitdom8 commented 1 year ago

Hi, can we run training in nodejs?