search

fboerncke / bloom-ai-simple-starter-nodejs

Simple starter BLOOM example showing how to access the web api to get something up and running in short time.
MIT License
31 stars 7 forks source link

Offline version, how to? #1

Closed linonetwo closed 1 year ago

linonetwo commented 1 year ago

Hi, do you know any solution that can run language model on nodejs without internet access?

I'm writing a electronjs based tiddlywiki app https://github.com/tiddly-gittly/TidGi-Desktop that is offline-first, and I want to make it possible to run a large language model (around 1GB is ok I think?).

fboerncke commented 1 year ago

That would be interesting indeed.

Unfortunately I never tried to get something like this up and running so I cannot help you here.

There are offline versions for stable diffusion (I am running this myself) available on GitHub, but those use pre trained models.

linonetwo commented 1 year ago

I just want to use pre trained language model, and fine turn locally, because I don't know much about machine learning frameworks. Maybe I can find some GPT based things... Thanks anyway.