kohjingyu / fromage

🧀 Code and models for the ICML 2023 paper "Grounding Language Models to Images for Multimodal Inputs and Outputs".
https://jykoh.com/fromage
Apache License 2.0
466 stars 34 forks source link

Hello, I wanna konw the purpose of create_image_of_text #11

Closed SZhanZ closed 1 year ago

SZhanZ commented 1 year ago

Hello, I notice that there is a method called 'create_image_of_text' in utils.py. I can be aware of its function to draw a picture with the text caption, but cannot understand its role in the code where it is be called. Could you plz explain the purpose to set it? Thanks in advance~

image
kohjingyu commented 1 year ago

Actually, this code is just for visualization purposes, it writes the image and the generated caption into Tensorboard outputs (https://github.com/kohjingyu/fromage/blob/main/fromage/evaluate.py#L180-L190). It is not used in the forward pass of the model.

Hope that makes sense!

SZhanZ commented 1 year ago

okay~ thanks alot