search

mecarill / classawareteacher

6 stars 2 forks source link

Error: Training has diverged #2

Open andyj1 opened 3 months ago

andyj1 commented 3 months ago

Hi, Thanks for the wonderful work.

From running the code straight out of the repo, I am encountering an error while running proposal generator: FloatingPointError: Predicted boxes or scores contain Inf/Nan. Training has diverged. Are the hyperparameters uploaded to this repo the correct setting as in the paper?

Any thoughts on this? Thanks.

mecarill commented 1 month ago

Thank you for your interest in our work! Are you running the CS>Foggy benchmark?

andyj1 commented 1 month ago

Thank you for your interest in our work! Are you running the CS>Foggy benchmark?

Thanks for your response! Yes, I ran into that issue while running C2F benchmark. It stops at a random iteration with that issue.

fnsfd commented 1 month ago

Thank you for your interest in our work! Are you running the CS>Foggy benchmark?

Thanks for your response! Yes, I ran into that issue while running C2F benchmark. It stops at a random iteration with that issue.

I retrained at the previous node, and this error disappeared . C2F

fnsfd commented 1 month ago

FloatingPointError: Predicted boxes or scores contain Inf/Nan. Training has diverged. Hello, I encountered this error while training on P2C. How can I solve it? Any suggestions would be greatly appreciated