search

anuragranj / flowattack

Attacking Optical Flow (ICCV 2019)
https://flowattack.is.tue.mpg.de
Other
59 stars 16 forks source link

FlowNetC Normalization & Weights #4

Open SimonSchrodi opened 3 years ago

SimonSchrodi commented 3 years ago

Hi,

I have a couple of questions regarding flowattacks on FlowNetC:

anuragranj commented 3 years ago

We found that it barely affected results, it was within 1% or so. When you subtract the mean and divide by max, assuming all images have the same range [0,1], or [0,255]; it would result in the same image tensor within [-1, 1] after normalization.

Also, it is very much possible that the numbers would be different. We noticed a really large variance in the numbers with different seeds. So we ran the training for the patch several times and picked the best attack for each.

Now, if your objective is to get higher numbers for attack, you could just remove augmentations. But then, when you remove augmentations, the attack doesn't generalize to real world. There is a trade-off and you can vary the amount of augmentation, or you can completely remove the augmentation by using the flag --norotate, to get higher numbers. We saw much higher numbers with no augmentations; but haven't used them in the paper.

Also, we used pretrained weights.

SimonSchrodi commented 3 years ago

Thanks for the answer.

So I noticed this too. However, difference is a bit larger for me, i.e. 11.5 unattacked EPE (using mean/max) and 14.52 unattacked EPE (normalize to [-1,1]) on KITTI 2015 train (without attack). Note that using mean/max is closer to the reported EPE (11.36) in the FlowNet2 paper.

How many runs did you try to obtain the "best" adversarial patch?

How can this be the same using mean & max? If I am not mistaken mean and max then would be both 0.5. However, this would not make too much sense, since basically the image then would be just a gray scale image. I think it is more in the range of [-0.4, 0.6] (this also corresponds to FlowNet2C's internal data normalization).

anuragranj commented 3 years ago

Okay, i think that's a valid issue. We did get around 14 EPE on unattacked FlownetC yes. We did at least 5-10 runs for each.