search

RobinDeSmet / BazelBot

This is a discord bot that generates non-sensical gibberish for entertainment purposes.
MIT License
0 stars 0 forks source link

Experiment with smaller models #15

Open RobinDeSmet opened 3 months ago

RobinDeSmet commented 3 months ago

The inference time is way to high, we should try to use a much smaller model from Ollama:

RobinDeSmet commented 3 months ago

Those smaller models do not have enough power to grasp the instructions in the prompt. They are however much faster. If such a model exists that is powerful enough for this task, we should definitely switch! For now stick with: gurubot/llama3-guru-uncensored:latest.