search

nnzhan / Graph-WaveNet

graph wavenet
MIT License
628 stars 201 forks source link

About inverse_transform #4

Closed Davidham3 closed 5 years ago

Davidham3 commented 5 years ago

Hi, I have run your code on some datasets that I constructed, and I found an problem in computing the MAPE metrics.

For the ground truth data, you normalize them into a small scale, and do the inserver normalize when evaluating the model. That will change some value like 0 to a small value like 1e-5. The computation of MAPE will take that small value into account and generate a very large MAPE value because mask only filter the value equals zero instead of these small value.

gcorso commented 5 years ago

Have you tried to mask all the values below a certain threshold (e.g. 1e-4) rather than just the ones equal to 0? Equality with floating point numbers is always tricky...

Davidham3 commented 5 years ago

@gcorso Yes, I know masking all values below a certain threshold can solve this problem of course, but I suggest using the original ground truth to evaluate the model rather than values that have been transformed two times.

gcorso commented 5 years ago

That's a good point @Davidham3, it actually seems like the model never uses the transformed labels, therefore not transforming them in the first place would save computation and avoid floating errors.

nnzhan commented 5 years ago

@Davidham3 @gcorso Thanks for finding out this issue. I have updated our codes, which does not transform the ground truth in the beginning.