Open WenaoMA opened 1 month ago
Hi,
I will upload the hyperparameters used for training the phi-3 mini and llama3-8b recently in one week. In my view, the difference is only the batch size in scripts, which has little influence.
You can check the REC task on my online demo, which is normal. If you find your training abnormal, I suggest you check the data in your datasets first. Please make sure the text or data fed to the model is correct. For example, your training text is 6 values.
I will check my implementation of the REC task and look forward to your updated hyperparameters. Thank you for your response!
Really nice work! Could the author also provide the hyperparameters used for training the phi-3 mini backbone (i.e., pretrain.sh and finetune_lora.sh)?
In addition, during my training process, I noticed an issue where the trained model outputs an incorrect number of coordinates for the Referring Expression Comprehension task. It should output 6 values, but the model I trained using finetune_lora.sh only outputs 4 values. Do you have any suggestions for this problem? Thank you!