search

uber-research / LaneGCN

[ECCV2020 Oral] Learning Lane Graph Representations for Motion Forecasting
https://arxiv.org/abs/2007.13732
Other
496 stars 131 forks source link

Can you tell me the reason for randomness? #30

Open noparkee opened 2 years ago

noparkee commented 2 years ago

Hello, Thanks for your nice project.

I tried to train the model several times without code editing, then I found that the difference in performance of each trial was quite large. There is no big difference when testing multiple times with one checkpoint, so randomness seems to occur during the learning process. Do you have any idea what could be the reason?

Thank you!

FUTUREEEEEE commented 2 years ago

i think the randomness is caused by the usage of nondeterministic-algorithms , which means even you set all the random seed ,the result will still have some randomness. check it out by add torch.use_deterministic_algorithms(True) into the main function .

FUTUREEEEEE commented 2 years ago

Hello, Thanks for your nice project.

I tried to train the model several times without code editing, then I found that the difference in performance of each trial was quite large. There is no big difference when testing multiple times with one checkpoint, so randomness seems to occur during the learning process. Do you have any idea what could be the reason?

Thank you!

btw,have you try to submit the test result to eval AI ? how is the outcome? i try to submit it result using the pretrained model provided by the author and my own training result , it turns out very bad....

noparkee commented 2 years ago

Thank you for you answer. I'll check it later. I didn't submit the evaluation, just checked the validation result. I am sorry for not being helpful 😅

huanmanlou commented 1 year ago

Hello, Thanks for your nice project. I tried to train the model several times without code editing, then I found that the difference in performance of each trial was quite large. There is no big difference when testing multiple times with one checkpoint, so randomness seems to occur during the learning process. Do you have any idea what could be the reason? Thank you!

btw,have you try to submit the test result to eval AI ? how is the outcome? i try to submit it result using the pretrained model provided by the author and my own training result , it turns out very bad....

I recently obtained the h5 file using the pretrained model provided by the author, but the results are much worse than what the author described in the paper. Do you know the reason? Thank you