Open Apollo1840 opened 2 years ago
Probably, e
in the loss formula in the paper actually stands for the z_q(x)
. The author did not write as z_q(x)
because its calculation evolves argmin, which is not-differentiable. However, this is not a problem to implement it naively as z_q(x)
, because tensorflow, as well as pytorch, will stop the gradient before argmin operation, thus it works as intended and causes no BUG.
That is my new understanding.
Probably,
e
in the paper stands for thez_q(x)
. The author did not write asz_q(x)
because its calculation evolves argmin, which is not-differentiable. However, this is not a problem to implement it naively asz_q(x)
, because tensorflow, as well as pytorch, will stop the gradient before argmin operation, thus it works as intended and causes no BUG.That is my new understanding.
please close this if admin thinks this explanation is right.
FIrst of all, maybe it is my misunderstanding of the paper, so hope somebody could explain it for me, thanks! :
in the paper, the loss is defined as![Screenshot from 2022-08-30 11-52-26](https://user-images.githubusercontent.com/28777429/187407855-e36af2f8-58c1-497d-b425-6fb3842ebe91.png)
where![Screenshot from 2022-08-30 11-57-36](https://user-images.githubusercontent.com/28777429/187408189-1dd44b68-d601-4b66-97d7-f726c5895cbe.png)
e
is the codebook defined at the beginning of the Section:So, in the paper, the codebook loss and commitment loss are MSE between
z_e(x)
ande
.However, in the implementation, they are implemented as MSE between![Screenshot from 2022-08-30 11-58-19](https://user-images.githubusercontent.com/28777429/187408392-b4a2fbfd-d0d6-42fb-8f21-45beb2de6937.png)
z_e(x)
(inputs) andz_q(x)
(quantized), where variable quantized means quantized encoding of the image, namelyz_q
:Are they actually the same thing? why?
z_e(x)
(H' W' D) ande
(K * D)?z_q(x)
(quantized) backprop since its calculation contains argmin?