Open qwedaq opened 3 months ago
@qwedaq sorry can you confirm which sample your running from the cookbook?
@qwedaq sorry can you confirm which sample your running from the cookbook?
Hi @leestott, I am running the following script for Phi3 Vision from the cookbook
@ChenRocks please can you look into this with your finetuning sample
Hi @qwedaq, thanks for reporting your results. Note that all deep learning training has inherent randomness; therefore, it is possible that a re-run results in slight accuracy difference.
However, in your case, the drop is significant. The reason is this --lora_alpha_ratio=16
hyper parameter. The correct way of setting lora_alpha
to 16
is --lora_alpha_ratio=0.5
. See this line.
I know this may not be obvious for users. I will improve the document later. Thanks!
Hi @qwedaq, thanks for reporting your results. Note that all deep learning training has inherent randomness; therefore, it is possible that a re-run results in slight accuracy difference.
However, in your case, the drop is significant. The reason is this
--lora_alpha_ratio=16
hyper parameter. The correct way of settinglora_alpha
to16
is--lora_alpha_ratio=0.5
. See this line.I know this may not be obvious for users. I will improve the document later. Thanks!
This is working now. I am able to reproduce the results. Thank you
I just had quick question related to the same code. I would like to know why Phi3V reports the final results using ANLS metric and does not use more modern metrics such BLEU, BERT or ROUGE-L?
This issue is for a: (mark with an
x
)Minimal steps to reproduce
Any log messages given by the failure
Expected/desired behavior
OS and Version?