A few models to consider for the ai model and a few links to go with them

[aaronstevenson408]

state of the art model in gguf format https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF

the library to run the model https://github.com/ggerganov/llama.cpp (look at bindings also for js bindings)

transformers.js( js version of transformers library)(you may not need it) https://huggingface.co/docs/transformers.js/en/index

open source visual language model ( for seeing the result): https://huggingface.co/NousResearch/Obsidian-3B-V0.5

[shiffman]

Thanks for these suggestions and research!

[shiffman]

Thank you for these suggestions! Starting out with https://ollama.com/.

These other models are great, but my goal is to use an on device model either via transformers.js or a local server on my M1 macbook.

[aaronstevenson408]

You might also wanna look at as a frontend lmstudio

All of the models i mentioned should be able to run on an m1 mac locally , some quantizations can even be run an a raspberry pi

[aaronstevenson408]

Also i believe the nous hermes 2 regular is available in ollama

[shiffman]

I'm going to close this issue, let's discuss specific models as needed! Right now I'm using

• llama 70b-chat
• whisper
• a few TTS options