implement weighted overlap loss function - Githubissues

arcee-ai / DAM

30 stars 4 forks source link

implement weighted overlap loss function #39

Closed thomasgauthier closed 2 weeks ago

thomasgauthier commented 3 weeks ago

Implemented @ElliotStein weighted overlap loss function.

I trained a merge (devout-firebrand-278) with it on the japanese/math task: https://wandb.ai/arcee-ai/Dynamic%20Adaptive%20Merging/runs/sllygkhu

Eval

Unfortunately it does not seem to beat similarity loss (proud-snowflake-238)