search

huanghoujing / beyond-part-models

PCB of paper: Beyond Part Models: Person Retrieval with Refined Part Pooling, using Pytorch
331 stars 81 forks source link

About the issue on training model by combined dataset #21

Closed Reeshark closed 6 years ago

Reeshark commented 6 years ago

When I training at the step 560, I have a error: Step 20/Ep 1, 2.56s, loss 45.9606 Step 40/Ep 1, 2.56s, loss 44.4427 Step 60/Ep 1, 2.57s, loss 43.4606 Step 80/Ep 1, 2.57s, loss 42.1291 Step 100/Ep 1, 2.57s, loss 42.6228 Step 120/Ep 1, 2.56s, loss 41.2495 Step 140/Ep 1, 2.57s, loss 39.7805 Step 160/Ep 1, 2.57s, loss 38.2572 Step 180/Ep 1, 2.56s, loss 38.0040 Step 200/Ep 1, 2.58s, loss 37.0701 Step 220/Ep 1, 2.57s, loss 36.9859 Step 240/Ep 1, 2.56s, loss 31.4023 Step 260/Ep 1, 2.57s, loss 34.4426 Step 280/Ep 1, 2.56s, loss 33.4842 Step 300/Ep 1, 2.57s, loss 32.7389 Step 320/Ep 1, 2.56s, loss 33.1874 Step 340/Ep 1, 2.57s, loss 30.5525 Step 360/Ep 1, 2.57s, loss 32.8995 Step 380/Ep 1, 2.58s, loss 30.6072 Step 400/Ep 1, 2.57s, loss 29.6854 Step 420/Ep 1, 2.55s, loss 27.2513 Step 440/Ep 1, 2.56s, loss 29.8557 Step 460/Ep 1, 2.56s, loss 30.4018 Step 480/Ep 1, 2.56s, loss 29.9007 Step 500/Ep 1, 2.56s, loss 27.1777 Step 520/Ep 1, 2.56s, loss 27.2997 Step 540/Ep 1, 2.57s, loss 27.3183 Step 560/Ep 1, 2.56s, loss 25.4691 Ep 1, 1476.23s, loss 34.8283 Traceback (most recent call last): File "script/experiment/train_pcb.py", line 498, in main() File "script/experiment/train_pcb.py", line 469, in main mAP, Rank1 = validate() File "script/experiment/train_pcb.py", line 389, in validate if val_set.extract_feat_func is None: AttributeError: 'TrainSet' object has no attribute 'extract_feat_func'

huanghoujing commented 6 years ago

Hi, thank you for your interest. I did not separate a val set for the combined dataset. So normally there shouldn't be validation during training on the combined dataset.

Reeshark commented 6 years ago

Thanks for reply. I will remove the val part.