How does PGDAttack deal with embedding？

[XueAdas]

Hi, It is a great open-source project! But as a beginner, I had some problems. In Pytorch version, delta in PGDAttack as a parameter should be type FloatTensor. But my model uses embedding layer, it has to transform delta into LongTensor, so delta will lose its gradient because only floatTensor in Pytorch can grad. What should I do?

[gwding]

Thanks for your interest in advertorch :) I'm not sure if I fully understand your question. Do you mean because your model has an non-differentiable component, you cannot get gradient so PGDAttacck will give you an error? If you can paste your code snippet and error message that'll be helpful.

[XueAdas]

Hi, I am glad to receive your timely reply! ^_^ When the program runs to line 65 of iterative_projected_gradient.py

for i in range(nb_iter):
    outputs = predict(xvar + delta)  # Here!
    loss = loss_fn(outputs, yvar)
    if minimize:
        loss = -loss

---

Tensor(xvar + delta) is supposed to be FloatTensor, but as the tensor is the input of the embedding layer in my model, I have to transform it into LongTensor in Pytorch, If I do this, it will result in an error like this line 79. So I don't know what to do?

    elif ord == 2:
        grad = delta.grad.data      # Error in here!
        grad = normalize_by_pnorm(grad)
        delta.data = delta.data + batch_multiply(eps_iter, grad)
        delta.data = clamp(xvar.data + delta.data, clip_min, clip_max
                           ) - xvar.data
        if eps is not None:
            delta.data = clamp_by_pnorm(delta.data, ord, eps)

---

Error message: 'NoneType' object has no attribute 'data'

[gwding]

Maybe you can have a look at the so called BPDA method and the BPDA wrapper in advertorch? https://github.com/BorealisAI/advertorch/blob/master/advertorch/bpda.py https://github.com/BorealisAI/advertorch/blob/master/advertorch_examples/tutorial_attack_defense_bpda_mnist.ipynb

Basically you can wrap your "transformation" module, so that it has gradient.

[gwding]

Closing the issue for now. Will reopen if problems remain.