search

meetdavidwan / crg

PyTorch code for "Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training"
https://contrastive-region-guidance.github.io/
MIT License
28 stars 1 forks source link

How to set generation_config #1

Closed lwpyh closed 7 months ago

lwpyh commented 7 months ago

Hi there,

Thank you for this great work, just want to know how to set the generation_config to trigger CFG when model_kwargs_blackout is supplied?

I have set generation_config=None or generation_config=model.generation_config, but both of them cannot trigger CFG, could you help me to address it?

Best, Jian

meetdavidwan commented 7 months ago

Thanks for your interest in the work!

Could you provide more information on what arguments you include for model_kwargs_blackout? How the code works should be just a single model_kwargs_blackout with some of the inputs with suffix _blackout. To trigger CFG, you would need (1) have the correct inputs similar to here which are processed in here, and (2) only have greedy decoding option (i.e. sample=false, num_beams=1). If you want to do sample, feel free to change the contrastive_generate according.py similar to how I changed greedy_decoding function.

Let me know if you have any further questions.

lwpyh commented 7 months ago

Thank you for your kind reply, I have addressed this issue, I think I have set the wrong setting since I am trying to modify the code on other decoding options. Thank you for your kindly reply.