search

BrutalCoding / shady.ai

Making offline AI models accessible to all types of edge devices.
https://www.shady.ai/
GNU Affero General Public License v3.0
126 stars 11 forks source link

[BUG] 🐛 - Missing a tokenizer. Probably need to add Dart bindings for HF tokenizer? #24

Closed BrutalCoding closed 1 year ago

BrutalCoding commented 1 year ago

Describe the bug I couldn't wrap my head around last couple of days on what's missing in order to have a conversational AI. A short while ago, I added the AI evaluation feature and successfully performed an eval on a 3B model. In other words, I was able to load it in and run a test against it.

Great, eval done. Chat can be done in less than 15 minutes of work. Right? Yes but no. You see, the problem is that I'm missing an important component: a tokenizer. I've been too blind to notice this until today thanks to a good friend of mine.

Now, that's the theory at least. Will put this to practice and see if I can finish it this week.

PS. I'm not an AI expert, more of an AI noob that's good enough to grasp concepts and simply able to read instructions to run them.

BrutalCoding commented 1 year ago

Closing now because my recent changes includes llama.cpp directly which comes with a tokenizer built-in.

Hooray!