Reproduce Lora results is close but not accurate

microsoft / LoRA

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

https://arxiv.org/abs/2106.09685

MIT License

10.43k stars 669 forks source link

Reproduce Lora results is close but not accurate #165

Open harsh306 opened 6 months ago

harsh306 commented 6 months ago

Getting this result

For these Hyperprams:

https://github.com/microsoft/LoRA/tree/main/examples/NLG#replicating-our-result-on-e2e

What were the hyperparameters for results in the paper: https://github.com/microsoft/LoRA/tree/main/examples/NLG#adapting-gpt-2-using-lora

qwertyczx commented 6 months ago

Hi, I met similar problem with you.This is my result: BLEU: 0.6865 NIST: 8.6808 METEOR: 0.4638 ROUGE_L: 0.7116 CIDEr: 2.5030 I reproduce it with V100-32g, following steps in: https://github.com/microsoft/LoRA/tree/main/examples/NLG#replicating-our-result-on-e2e

RayCyder commented 4 months ago

me too

xiaoshingshing2 commented 1 month ago

I have the same problem. This is my result: BELU: 0.6885 NIST: 8.68 METEOR: 0.4638 ROUGE_L: 0.7139 CIDEr: 2.5079 I reproduce it with A100-40g, following exactly the same process as in readme.