cleverhans-lab / cleverhans

An adversarial example library for constructing attacks, building defenses, and benchmarking both
MIT License
6.15k stars 1.39k forks source link

bug in CarliniWagnerL2 for tf2 #1205

Open grahamannett opened 3 years ago

grahamannett commented 3 years ago

Seems like this is because CarliniWagnerL2.attack takes a single instance of x? Not sure if this is intended but is there anyway to allow for multiple x to be passed in at once? Would that help with the speed of the attack since at the moment generating a single attack for an in put takes ~45 seconds on a pretrained model.

The issue tracker should only be used to report bugs or feature requests. If you are looking for support from other library users, please ask a question on StackOverflow.

Describe the bug when trying to use cw2 attack from cleverhans I get an error about

ValueError: Dimensions must be equal, but are 3 and 10 for '{{node mul_5}} = Mul[T=DT_FLOAT](const, Maximum)' with input shapes: [10,160,160,3], [10].

To Reproduce Steps to reproduce the behavior:

  1. Using dataset from fastai IMAGENETTE dataset
  2. Using functional api from tf.keras with a pretrained model
  3. Something like this
    attacker = CarliniWagnerL2(model)
    adv_x = attacker.attack(x_tensor)

results in

    ValueError: Dimensions must be equal, but are 3 and 10 for '{{node mul_5}} = Mul[T=DT_FLOAT](const, Maximum)' with input shapes: [10,160,160,3], [10].

Where the issue is, is from (I think): https://github.com/cleverhans-lab/cleverhans/blob/master/cleverhans/tf2/attacks/carlini_wagner_l2.py#L333

where const is a tensor from tf.ones * initial const of the x input and loss_1 is a tensor that is the shape of the y output. I don't exactly see where this value comes from in the original paper so unsure exactly what the value should look like

Expected behavior Adversarial output

Screenshots If applicable, add screenshots to help explain your problem.

akshaymehra24 commented 2 years ago

Change in line https://github.com/cleverhans-lab/cleverhans/blob/master/cleverhans/tf2/attacks/carlini_wagner_l2.py#L153

From: const = tf.ones(shape) * self.initial_const

To: const = tf.ones(shape[0]) * self.initial_const

thijs-vanweezel commented 7 months ago

To: const = tf.ones(shape[0]) * self.initial_const

Could we please implement this in the main branch?