Closed ShogoAkiyama closed 5 months ago
Thank you for being so careful. I already corrected the code three months ago, see Line 51.
@ChenYi99
Thank you for your response.
Upon review, I also missed the conversion of the video to RGB.
However, separate from the video, since the current image is being used as input, I think a similar RGB conversion process might be necessary before liine 63
as well. I tried this in my environment, and it also improved the accuracy of Ego4D(out-domain) with the model which you shared.
https://github.com/ChenYi99/EgoPlan/blob/2a32be67434f920dc0b241dfa5280b2840445e7e/src/egoplan_video_llama_interface.py#L63
Could you please try it out when you have time?
Thank you very much.
Thanks for your feedback. I have updated the code according to your comments.
@ChenYi99
Hello,
I've encountered an issue when loading images for Ego4D where it seems the images are not being converted to RGB format. Specifically, I'm referring to this line in your code: https://github.com/ChenYi99/EgoPlan/blob/2a32be67434f920dc0b241dfa5280b2840445e7e/src/egoplan_video_llama_interface.py#L63
I managed to resolve the issue by adding the following line at line 62:
After applying this modification and testing your fine-tuned Video-LLaMA on EgoPlan-Bench Ego4D(out-domain), I noticed an improvement in accuracy from 41% to 43%. I was wondering if you could take a moment to review this modification.
Thank you for your time and assistance.