search

pratyushasharma / laser

The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction
https://pratyushasharma.github.io/laser/
MIT License
365 stars 28 forks source link

What is the ETA on the code #2

Closed MrigankRaman closed 10 months ago

MrigankRaman commented 10 months ago

Thanks for the awesome work. I would like to try it out. What is the ETA on the code?

dkmisra commented 10 months ago

A development release is available now. We will release a major refactoring in the first half of January to make the code very easy to apply to a new domain in BigBench and other popular LLM benchmark.

If you have any questions about using the development, then please let us know. For now, there is a file for each LLM and dataset combination and you just need to pip install the requirements and then run these files and providing the lname, lnum, and rate hyperparameters. The only place which will need some work for some files is that you need to do hyperparameter selection on the validation set separately.

dkmisra commented 10 months ago

Closing this thread. Feel free to re-open the thread. We welcome you to checkout the refactored version in mid-January, and if you have questions about this developmental version then either raise a new issue or email us. Thanks.