Open mkrupczak3 opened 10 months ago
Removing the negative training images seems to have had little impact. Same result as before.
At our meeting today, discussed how this might be because the patch training dataset is much too large. Theory is that with so many different images, the model is having a hard time learning what works and what doesn't with any single change to the patch.
Going to try a training run on a much smaller subset of data (about 8 images) and if it does any better
Here was the result after 5000 epochs on a set of 8 images:
Not enough time in the oven. Doing another run over the weekend, this time of 500,000 epochs to see if it works
Hello, I would like to ask why the patch generated by running the code has no gradient. It means that the patch has no changes after multiple rounds of training.I am using the pedestrian data set and the original patch is gray。
It's a known issue, the way we are dealing with the tensors is a bit flawed. There's some significant architectural differences between YOLOv4 and YOLOv8 we haven't been able to work out yet
Here was the result after 5000 epochs on a set of 8 images:
Not enough time in the oven. Doing another run over the weekend, this time of 500,000 epochs to see if it works
This didn't do any better; issue is something with the tensors
So, does this mean that your project cannot successfully train adversarial examples?
At present, yes. @SlicedBacon has been making some progress but I don't think it's been published here yet
That's unfortunate, but thank you.
Currently, patch training is failing to generate an adversarial pattern. It's just optimizing for non-printability score (nps) by generating an image of a single color:
@SlicedBacon and I believe this is because the Airbus dataset has negative training images (without objects present) which interfere with the patch training's stochastic gradient descent.
The original paper (and thus, the code repo this one is based on) used the INRIA dataset for person detection. @SlicedBacon noted this dataset does not have any negative training images where a person is not present.
In the current patch training code, any images without objects present will generate a
det_loss
of 0.0, giving the optimizer a false signal that the patch completely defeated the object detector. This seems to be interfering with the patch training process, causing the discontinuities observed in thedet_loss
graph from tensorboard shown above.For a quick fix, we may remove the negative training images from the dataset when using it for patch training. We should fix it properly so it ignores negative training images in the future though so other researchers can use our code