Closed charismaticchiu closed 2 months ago
Hi @charismaticchiu,
Thank you for your interest in our work! To address your questions:
I hope this helps! If you have any other questions, feel free to ask.
Thanks for the quick reply! Make sense!
Btw, where can I find the question file in this line?
Awesome, thanks!
and in the paper the model is trained 4 epochs, but the script shows it was trained 10 epochs, was that a typo? Also I am a bit confused by the steps, which overrides the num_train_epochs
. Does that match the implementation details right before sec 3.2?
And do you mind sharing the script for using LLava-Next as supervision model?
max_steps
to control how many training/optimization steps are needed, rather than num_train_epochs
. Actually, we also conducted some further experiments with this codebase before releasing. So the default hyper-parameters written in scripts might not all match what shown in the paper. For reproduction, we refer users to the detail introduction in the paper.I want to know why the max_steps is set to 2672? Does it have any special meaning?
No, it does not have any special meaning, it is just because the number of steps per epoch is not a pretty number like 1024 or 1500. Feel free to open a new issue if you have any questions.
Hi 2 quick questions,
From the paper algorithm1, I get a sense that the algorithm can work in an online divide-n-conquer manner with updated model and I am just curious when the self-feedback code will be released.
In this line the reference model is initialized the same as policy model (weights and dtype), so why wouldn't the log_p be the same and thus the loss be 0 all the time?
Thank you!