Roadmap: Integrate with pyvene and pyreft

[AriMKatz]

Both in Stanford nlp GitHub org. I haven't done experiments comparing them to pure prompt engineering, but my sense is that there will be places each one excels, particularly with very granular changes, deep or robust steering (like gokden gate claude) and perhaps an ensemble will give sota in many situations. Also don't have to worry about Prompt leakage.

Pyreft might need special user space API but pyvene can just accept xy pairs

[okhat]

Hey @AriMKatz ! Thanks for the suggestion! This would probably fit as an implementation of BootstrapFinetune. I perceive PyREFT as a form of finetuning, and I imagine it's probably less expressive from a quality standpoint in our setting than just doing full finetuning? Of course, it has other advantages like parameter efficiency.

[AriMKatz]

Hi @okhat ! Yes, I agree and I switched them. Meant to say pyreft is relatively straightforward with x,y pairs. And yea probably less expressive.

For pyvene it's probably a whole research project to compile high level feedback to specific interventions.