search

ggerganov / llama.cpp

LLM inference in C/C++
MIT License
65.29k stars 9.35k forks source link

How to make RLHF? #3561

Closed yukiarimo closed 5 months ago

yukiarimo commented 11 months ago

For example consider this model: https://huggingface.co/TheBloke/Pygmalion-2-7B-GGUF

I want to make (good/bad) RLHF like this:

User: Which drug do you like? AI: I like Meth (say this bad, and maybe provide good response) User: What is your name? AI: Iā€™m ChatGPT (say this is good, and provide another good response, or not)

Then using this knowledge fine tune LoRA or Model

BarfingLemurs commented 11 months ago

Most UI developers probably already add a šŸ‘ šŸ‘Ž , where šŸ‘ saves your input output. Once you 1000 šŸ‘ , you train saved messages

yukiarimo commented 11 months ago

Are you talking about šŸ‘ šŸ‘Ž in ChatGPT or Character.AI? Because as far as I know in the open source UIs like ooba or TavernAI, etc, there no ranking (even fine-tuning guides are not well-explained yet)

github-actions[bot] commented 5 months ago

This issue was closed because it has been inactive for 14 days since being marked as stale.