Question on Eq. 4

[denkorzh]

Hi, thanks for sharing your results!

I'm afraid I did not really get how you derived Eq. 4: if I'm not mistaken, ∇ p(c ∣ xₜ) = − λ ∇ ℰ (c, xₜ) + λ ∇ 𝔼 [ ℰ (c, xₜ) ], where the gradient ∇ is w.r.t. xₜ, and the expectation 𝔼 is over p(c ∣ xₜ).

Why have you decided to ignore the second term? Thank you in advance!

[vvictoryuki]

Hi, @denkorzh Thanks for your attention! I get the Eq.4 just considering the Z in Eq.3 is a constant, then we discard this term after computing the $\nabla_{\mathbf{x}_t}\log(\cdot)$. Could you tell me how you derived this expectation term? Perhaps we can find some better insights from it to help us improve our work! Look forward to your reply:)

[xjtupanda]

Hi, @denkorzh Thanks for your attention! I get the Eq.4 just considering the Z in Eq.3 is a constant, then we discard this term after computing the ∇xtlog(⋅)∇xtlog⁡(⋅)\nabla_{\mathbf{x}_t}\log(\cdot). Could you tell me how you derived this expectation term? Perhaps we can find some better insights from it to help us improve our work! Look forward to your reply:)

@vvictoryuki I have the similar confusion. Writing the normalizing constant $Z=\int_c exp(-\lambda \mathcal{\epsilon(c, xt)}) dc$ in expectation form, we have: $$ Z=\mathbb{E}{c \in p(c|x_t)} \ [\exp (-\lambda \mathcal{\epsilon(c, x_t)})] $$ Plugging into $\log p(c|x_t) = -\lambda \mathcal{\epsilon(c, x_t)} - \log(Z)$ and take derivatives on both sides: $$ \nabla \log p(c|x_t) = -\lambda \nabla \mathcal{\epsilon(c, x_t)} - \frac{\nabla \mathbb{E} [\exp (-\lambda \mathcal{\epsilon(c, x_t)})]}{Z} $$ But the term Z, more specifically the $\mathbb{E} [\exp (-\lambda \mathcal{\epsilon(c, x_t)})]$ term, if I don't get it wrong, is a function of $x_t$, so the gradient term should not be discarded?

[denkorzh]

@vvictoryuki Exactly, as @xjtupanda mentioned above, Z is a constant w.r.t. c but is a function of x_t.

[vvictoryuki]

Thank you for pointing out the shortcomings in our work. We will address the relevant inaccuracies in the latest arXiv version.