Laser Benchmarking on DPO Training Dataset with similiarity distance scoring

cognitivecomputations / laserRMT

This is our own implementation of 'Layer Selective Rank Reduction'

Apache License 2.0

229 stars 26 forks source link

Laser Benchmarking on DPO Training Dataset with similiarity distance scoring #13

Closed l4b4r4b4b4 closed 7 months ago

l4b4r4b4b4 commented 7 months ago

After noise matrix reduction model is tested against a number of configurable samples from a dpo training dataset.

The generated answers are scored against chosen/rejected dpo pairs. High cosine similiarity with chosen results in a high value being added to the overall performance score and higher similarity with rejected results in higher score substracted from the overall performance score.