Continue finetuning Qwen 2.5 7B for `evmind`

[brunneis]

Try to improve the baseline for Solidity generation.

[brunneis]

• Continued finetuning Qwen 2.5 7B with much more steps
• Several memory crashes after a number of steps (still training)

Next steps:

• Improve the baseline
• Finetune a base model with Solidity resources
• Try a contrast-based finetuning with ORPO
• Try reflection finetuning (Reflection 70B training data and model were released today, surprisingly good results)

[brunneis]

• The latest finetuning is "lazy". Produces a lot of placeholder comments such as "// TODO" or "// Your code here". Tried several custom and pre-existing LogitsProcessor implementations without relevant success. Trying ORPO to disincentivize the model to be "lazy" with a smaller synthetic dataset.

  • 84,869 rows
  • 1 epoch
  • Learning rate: 2e-4
  • Context: 8192
  • LoRA rank: 128
  • LoRA alpha: 128