search

radareorg / r2ai

local language model for radare2
https://www.radare.org
MIT License
69 stars 13 forks source link

Finetuning scripts #9

Open trufae opened 6 months ago

trufae commented 6 months ago

Extending base models with custom information like r2 source code, the book, disassembly data, decompilation code and more is interesting, so r2ai should provide the basic infrastructure to finetune models without the hassle to write code. This guide is quite comprensible and easy to follow.

https://medium.com/@mohammed97ashraf/your-ultimate-guide-to-instinct-fine-tuning-and-optimizing-googles-gemma-2b-using-lora-51ac81467ad2

dnakov commented 6 months ago

Yeah it'd be nice to facilitate that process but I think it'd be best if we provided a data set and something like an example for how to run a LORA fine tune, or even just a link to one. There are many other projects that are better suited to handle the fine tuning process. Fine tuning scripts get stale quickly, especially now with a lot of different variations of lora.

dnakov commented 6 months ago

On the other hand, it'd be good if we trained and maintained an r2 model and provided the weights. I have some AWS credits i don't mind burning for this but would need help preparing the dataset

trufae commented 5 months ago

Agree on that. What we should focus is on documenting and providing ways to generate all this training data in a way that can be consumed to finetune our own models. ideally the functionary one or the mistral. or even utopia are the ones im using the most and would love to improve.

trufae commented 5 months ago

https://www.kdnuggets.com/mistral-7b-v02-fine-tuning-mistral-new-open-source-llm-with-hugging-face