fungtion / DANN

pytorch implementation of Domain-Adversarial Training of Neural Networks
MIT License
841 stars 151 forks source link

Is the objective function of DANN defined incorrectly in the paper? #21

Closed ugurcanakyuz closed 1 month ago

ugurcanakyuz commented 2 months ago

The loss function in the paper, below, consists of a negative lambda (λ) value for domain loss. However, in your implementation, at this line, you sum all the loss values instead. Isn't this opposite of the loss defined in the paper?

image

But, in my opinion, summing up all the loss values as you did is correct because while the domain head tries to distinguish the domain and minimize its loss, the feature extractor tries to confuse it and maximizes its loss with the help of updates based on the GRL layer. If the negative lambda value was used in the loss, it would also confuse the domain head, and training would be unstable.

So, is the negative lambda value in the paper wrong, or am I missing something?

ugurcanakyuz commented 2 months ago

I was wrong :). The loss function was defined with this formula:

Screenshot 2024-08-14 111606

"We can then define the objective “pseudo-function” of (θf , θy , θd) that is being optimized by the stochastic gradient descent within our method:"

Yet, they define it with this formula in the Journal paper: image

I first started to read the journal paper. That made me confused.