toshikwa / fqf-iqn-qrdqn.pytorch

PyTorch implementation of FQF, IQN and QR-DQN.
MIT License
161 stars 24 forks source link

Could you please help me with the proof for proposition 1? #6

Closed xlnwel closed 4 years ago

xlnwel commented 4 years ago

Hi @ku2482

Thanks for the code. May I ask you a question?

The author gives proposition 1 and its proof as follows:

image

I'm quite confused about how they compute the third step, which involves the integral over a quantile function. Could you please help me with that?

toshikwa commented 4 years ago

Hi @xlnwel

I wrote the rough derivation of the first half(①). Note that

If you have any questions, feel free to ask me.

S__424837122

xlnwel commented 4 years ago

Hi @ku2482,

Thank you for the derivation, which helps me understand most of the proof. However, I still got several questions. The following derivative is what I wrote

image

The second and third parts are consistent with yours, but the first and fourth are different. I computed the first and fourth parts from

image

but you directly moved the partial derivative inside. May I ask you why?

This affects how to compute

image

which I calculate as follows

image

This is quite different from the results of proposition 1. Where did I make mistakes?

toshikwa commented 4 years ago

Hi.

Oh, you're right. Partial derivative should be placed as the second image you sent.

However, results would be the same. First and fourth parts are constants, which don't depend on omega. Because tau_hat is the midpoint and the signs of first and forth parts are opposite, these terms should disappear before we apply the partial derivative.

xlnwel commented 4 years ago

Hi, @ku2482

I know that. But could you please help me with the following calculation? I somehow compute the following part wrongly, and I cannot find the error

image

Thanks in advance.

toshikwa commented 4 years ago

Hi, @xlnwel

I'm not sure where exactly confuses you. Let me assure you that my derivation only calculates only first half of the whole derivative (①) as below, is it clear?

toshikwa commented 4 years ago

I wrote down the derivation of latter half of the whole derivative. Please tell me where exactly is bothering you.

Maybe, you wrongly removed the absolute value symbol, didn't you? Note that

Is everything clear?

S__425148434

xlnwel commented 4 years ago

Hi,

Everything is crystal clear now. Thank you so much for the proof:-)