search

vgel / repeng

A library for making RepE control vectors
https://vgel.me/posts/representation-engineering/
MIT License
435 stars 31 forks source link

Base model and vector merging #1

Open whitepapercg opened 6 months ago

whitepapercg commented 6 months ago

Hi, can you suggest how to merge the base model and the trained vector?

vgel commented 6 months ago

Hmm, not sure if it's possible since the library wraps extra code around the model to add the vector. Maybe by adding the vector to the appropriate layer bias? But it might be architecture-dependant based on normalization... I've been thinking about this as well so open to suggestions here!