Open peck94 opened 3 years ago
Hi,
thanks for bringing this up, I've never experimented with datasets with less than 4 classes. I think it makes a lot of sense to raise a warning stating that in such case AA can't be run as in the original version, I'll try to add it soon.
As a possible replacement, I think that the simplest choice would be falling back to the margin loss, although it's not scale invariant. I'll check whether better solutions exist (as I think) and possibly integrate some in the code.
The DLR loss is one of the major innovations of your work and is central to one of the four attacks used in the AutoAttack benchmark, APGD-DLR. However, when I was running tests with your framework on a couple of data sets, I noticed that AutoAttack had a tendency to crash when running the APGD-DLR attack. This is caused by the fact that the DLR loss function as defined in equation (6) of your paper implicitly assumes that the classification problem is composed of at least 3 classes; the targeted version presented in equation (7) assumes at least 4 classes.
This limitation raises a number of concerns which I think should be addressed: