search

pratyushasharma / laser

The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction
https://pratyushasharma.github.io/laser/
MIT License
365 stars 28 forks source link

how to get base model accuracy #21

Closed pursure-D closed 7 months ago

pursure-D commented 9 months ago

I should run which command to obtain the accuracy of the base model？

dkmisra commented 9 months ago

You can use the flag --lname dont to disable performing an intervention and, thefore, use the base model. This happens even if --rate and --lnum values are provided. See the implementation here:

https://github.com/pratyushasharma/laser/blob/main/src/laser/gptj_laser.py#L94 https://github.com/pratyushasharma/laser/blob/main/src/laser/llama2_laser.py#L100 https://github.com/pratyushasharma/laser/blob/main/src/laser/roberta_laser.py#L45 https://github.com/pratyushasharma/laser/blob/main/src/laser/phi1_5_laser.py#L89 https://github.com/pratyushasharma/laser/blob/main/src/laser/mujoco_dt_laser.py#L40

dkmisra commented 7 months ago

Closing this as there has been no further activity. Please feel free to open it, if needed.