showlab / Image2Paragraph

[A toolbox for fun.] Transform Image into Unique Paragraph with ChatGPT, BLIP2, OFA, GRIT, Segment Anything, ControlNet.
Apache License 2.0
789 stars 53 forks source link

Dense Caption always return empty #6

Closed mickelliu closed 1 year ago

mickelliu commented 1 year ago

Hi,

Thanks for sharing this work. Very interesting and potentially very impactful.

I encounter this issue while running python main.py --image_src "/Code/Image2Paragraph/examples/3.jpg" --out_image_name "output/3_result.jpg"

"Dense Cpation" always returns "/", and the program processes without error. I was able to get the generated text at the end along with the style-transferred image, but the caption is a bit off potentially due to the missing dense caption.

1

FingerRec commented 1 year ago

Hello Mickeliu,

The dependencies of the GRIT model are complex, which is why I have implemented a subprocess to call it. When the GRIT model runs, it outputs a lot of logs. In order to streamline the process and avoid overwhelming feedback, I have set it to run in 'quiet' mode.

To troubleshoot any issues, please ensure that you are able to run the GRIT demo independently and successfully. Additionally, check the environment name specified in the GRIT_model.py file. Finally, ensure that you are using an absolute path for your input image to avoid any potential file path errors.

Let me know if you have any further questions or concerns.

FingerRec commented 1 year ago

hi, we have integrated the grit into our code, no need to install grit independently anymore, follow install.md for details.

mickelliu commented 1 year ago

hi, we have integrated the grit into our code, no need to install grit independently anymore, follow install.md for details.

Thanks for letting me know and thanks for the hard work. I will check it out later.