search

AoiDragon / HADES

[ECCV'24 Oral] The official GitHub page for ''Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models''
MIT License
21 stars 3 forks source link

some question about the white box attack #10

Open yuese1234 opened 1 month ago

yuese1234 commented 1 month ago

Sorry to bother you again. When I run your white-box attack code for LLaVA, the gradient is None. The specific problem is: adv_noise.data = (adv_noise.data - alpha * adv_noise.grad.detach().sign()).clamp(0, 1) AttributeError: 'NoneType' object has no attribute 'detach'. Can you give me some help?

pyogher commented 4 days ago

Hi @yuese1234, could you please provide your versions of torch and transformers? Alternatively, you can use transformers 4.36.2 and torch 2.1.0.

yuese1234 commented 4 days ago

Hi @yuese1234, could you please provide your versions of torch and transformers? Alternatively, you can use transformers 4.36.2 and torch 2.1.0.

I use the transformers 4.37.2 and torch 2.1.2。I found that in the VIT of CLIP, the gradient disappeared, but I rewrote a piece of code and loaded the VIT separately, and the gradient existed.

pyogher commented 4 days ago

Hi @yuese1234, could you please provide your versions of torch and transformers? Alternatively, you can use transformers 4.36.2 and torch 2.1.0.

I use the transformers 4.37.2 and torch 2.1.2。I found that in the VIT of CLIP, the gradient disappeared, but I rewrote a piece of code and loaded the VIT separately, and the gradient existed.

Ok, we will update the version requirements in the README later. Please let us know if you have any other questions.

yuese1234 commented 4 days ago

Hi @yuese1234, could you please provide your versions of torch and transformers? Alternatively, you can use transformers 4.36.2 and torch 2.1.0.

I use the transformers 4.37.2 and torch 2.1.2。I found that in the VIT of CLIP, the gradient disappeared, but I rewrote a piece of code and loaded the VIT separately, and the gradient existed.

Ok, we will update the version requirements in the README later. Please let us know if you have any other questions.

I just try transformers 4.36.2 and torch 2.1.0. the gradient is still None. As the follows: images.requires_grad=True image_features = self.encode_images(images) print(f'image_features:{image_features.requires_grad}') the result is False . so crazy！