search

yunqing-me / AttackVLM

[NeurIPS-2023] Annual Conference on Neural Information Processing Systems
https://arxiv.org/pdf/2305.16934.pdf
MIT License
160 stars 8 forks source link

Minigpt4 code problerm #22

Open returnaaa opened 3 weeks ago

returnaaa commented 3 weeks ago

I can not understand why size=(batch_size, 577, 768), in vit , the size = (batch_size, 257, 1408), in vit and q-former , the size = (batch_size, 32, 768) image

sftsgly commented 2 weeks ago

bro , i can't find the function forward_encoder. Do you know how to find it?

returnaaa commented 1 week ago

bro , i can't find the function forward_encoder. Do you know how to find it? It seems that the author needs to implement it himself, but he doesn't seem to have open sourced it.

sftsgly commented 1 week ago

I wrote some code based on the answer function in demo.py to experiment, but the effect was not good. https://github.com/Vision-CAIR/MiniGPT-4/blob/main/demo.py

returnaaa commented 1 week ago

I wrote some code based on the answer function in demo.py to experiment, but the effect was not good. https://github.com/Vision-CAIR/MiniGPT-4/blob/main/demo.py

hi, bro, Can you share the code you modified, I have doubts about this function

sftsgly commented 1 week ago

Desktop.zip Here, but in query.py I delete the wandb. And this function can only pass one image at a time.