search

zh460045050 / VQGAN-LC

61 stars 6 forks source link

Clip feature in VQModel but not used #4

Open xesdiny opened 2 weeks ago

xesdiny commented 2 weeks ago

In vqgan-gpt-lc/eval_reconstruction.py line 430:431

...
     with torch.no_grad():
            _, _, _, _, _, tk_labels, xrec = model(x, clip_image.to(device), data_iter_step, step=0, is_val=True)

In vqgan-gpt-lc/models/model_vq.py line 223

...
        def forward(self, input, global_input, data_iter_step, step=0, is_val=False):

This global_input is not used in the forward func, but it is still passed into the method as a parameter. Are there other more interesting uses of this clip features?

zh460045050 commented 1 week ago

Thank you for your interest in our work. Currently, the CLIP features are solely used for codebook generation. In future updates, we will utilize the "global_input" in "eval_reconstruction.py" to evaluate the CLIP score.