princeton-vl / px2graph

Training code for "Pixels to Graphs by Associative Embedding"
BSD 3-Clause "New" or "Revised" License
133 stars 22 forks source link

Stuck in training #2

Closed panfengli closed 6 years ago

panfengli commented 6 years ago

I ran the training codes twice. The first time it got stuck at the 2nd 2000 iteration, and the second time it got stuck at the 3rd 2000 iteration.

100%|##########| 2000/2000 [47:19<00:00,  1.42s/it]
100%|##########| 100/100 [01:22<00:00,  1.22it/s]
100%|##########| 2000/2000 [45:40<00:00,  1.37s/it]
100%|##########| 100/100 [01:17<00:00,  1.29it/s]
34%|###3      | 670/2000 [14:58<29:42,  1.34s/it]

I am not sure why it happened.