reproduce results

[zouguojian]

Who can reproduce similar results on the METR-LA dataset? My result is worse than this paper.

[liyaguang]

Thanks for the interests in the work and the feedback. The following tips may help reproduce the results:

• Use the python packages with versions recommended in the Requirement section, ideally with tensorflow version 1.3.0 since I have not tested the training after that version.
• Check possible gradient explosion or adjust the learning rate as recommended in the training details section. There is a chance of training loss explosion, one temporary workaround is to restart from the last saved model before the explosion, or to decrease the learning rate earlier in the learning rate schedule.
• Here are the related tensorboard links when training the models, DCRNN on METR-LA, DCRNN on PEMS-BAY. You may use them as a reference, e.g., comparing the train/eval loss curves, eval metrics at different time horizons.

Hope this can help, and feel free to update this thread if you have encountered issues.

[zouguojian]

Yep, it does not work in the METR-LA dataset. Although I have used the same codes on the server, it is worse than the results you give. Of course, I have read the tips you provide, but it is non-helpful for me. I want to know how others' experimental results. Thank you for your reply.

[zouguojian]

[zouguojian]

it does not work

[zouguojian]

Attention, please; horizon 13 is the average result.

[liyaguang]

Thank you for providing the log, and spending time debugging this. It seems that there may be some discrepancies, as even a naive baseline approach considering constant traffic could yield significantly better accuracy. Additionally, would you kindly share the version of Tensorflow that you are currently using? It is important to note that the provided code has solely been tested with Tensorflow 1.x, and may not function properly with Tensorflow 2.x.

Attached is my training log from 2020 with Tensorflow 1.14.0, which I hope will be of assistance: https://gist.github.com/liyaguang/f689b004f6aa04f6767bbadaeb808d47.

Furthermore, I have conducted another run today, the results of which were quite comparable. In the following days, I plan to allocate time to test it on Tensorflow 2.0.

[zouguojian]

Dear Li, it's okay to test codes on the pre-trained model weights you give (i.e., ). However, I couldn't get the results when I trained the model myself. The version of TensorFlow is 1.14.

[zouguojian]

Thank you for providing the log, and spending time debugging this. It seems that there may be some discrepancies, as even a naive baseline approach considering constant traffic could yield significantly better accuracy. Additionally, would you kindly share the version of Tensorflow that you are currently using? It is important to note that the provided code has solely been tested with Tensorflow 1.x, and may not function properly with Tensorflow 2.x.

Attached is my training log from 2020 with Tensorflow 1.14.0, which I hope will be of assistance: https://gist.github.com/liyaguang/f689b004f6aa04f6767bbadaeb808d47.

Furthermore, I have conducted another run today, the results of which were quite comparable. In the following days, I plan to allocate time to test it on Tensorflow 2.0.

I have uploaded the new version to my personal page, https://github.com/zouguojian/Traffic-speed-prediction/tree/main/3S-TBLN/baseline/DCRNN. you can download these codes on your local device to run. looking forward your reply

[liyaguang]

Thank you for sharing the code. Would it be possible for you to confirm whether your can train unmodified code using Tensorflow 1.x (not compat model from 2.x)?

This code appears to be designed for Tensorflow 2.x in compat model rather than native 1.x. Since the DCRNN Cell implementation relies on the contrib library, which is not officially supported by Tensorflow and could potentially lead to backward compatibility issues. Therefore, it is important to ensure that the code on Tensorflow 1.x without modification to reproduce the results.

In the meantime, I will also investigate how to make the code work with Tensorflow 2.x (in compat model). Thanks for your kind understanding.

[zouguojian]

Thank you for sharing the code. Would it be possible for you to confirm whether your can train unmodified code using Tensorflow 1.x (not compat model from 2.x)?

This code appears to be designed for Tensorflow 2.x in compat model rather than native 1.x. Since the DCRNN Cell implementation relies on the contrib library, which is not officially supported by Tensorflow and could potentially lead to backward compatibility issues. Therefore, it is important to ensure that the code on Tensorflow 1.x without modification to reproduce the results.

In the meantime, I will also investigate how to make the code work with Tensorflow 2.x (in compat model). Thanks for your kind understanding.

Yep, dear Li, maybe you are right. I have used the high version of TensorFlow to be compatible low version.

[zouguojian]

I have solved this problem in PyTorch！！！

[liyaguang]

Great job! It's wonderful to hear that you were able to solve the problem using PyTorch! If you want to, feel free to share your solution with others

[zouguojian]

Great job! It's wonderful to hear that you were able to solve the problem using PyTorch! If you want to, feel free to share your solution with others of cause, I will public it in the next week