search

amoudgl / pygoturn

PyTorch implementation of GOTURN object tracker: Learning to Track at 100 FPS with Deep Regression Networks (ECCV 2016)
MIT License
135 stars 36 forks source link

Error when executing test.py which is similar to the error #17 #18

Closed fzh0917 closed 5 years ago

fzh0917 commented 5 years ago

The code is latest. The problem description and exception infomation are same as the issue #17. However, the code worked finely when using the pretrained model file which was downloaded from Google Drive. So, I am thinking that whether the number of batch is too small when training the model manually. P.S. My num_batch is 50 when training the model manually by executing the command python train.py.

amoudgl commented 5 years ago

I guess your training checkpoint may be regressing image coordinates which are totally out of image bounds. In the next frame, GOTURN crops the image at the previous predicted box, which could be causing this issue.

I tested the PIL crop method for some cases when your box had coordinates out of image but it seemed to work fine and padded the out of box area with zeros.

Can you check bounding box values? Just print self.prev_rect in the test method of test.py.

amoudgl commented 5 years ago

I'll look into this issue in detail and get back. Thanks!

fzh0917 commented 5 years ago

Here is the stack trace.

$ python test.py -w ../saved_checkpoints/exp3/final_model.pth 
Namespace(data_directory='../data/OTB/Man', model_weights='../saved_checkpoints/exp3/final_model.pth')
[69. 48. 95. 87.]
frame: 1 [69.91484090260097, 44.97729134559631, 94.00789887564522, 79.06419345310756]
[69.91484090260097, 44.97729134559631, 94.00789887564522, 79.06419345310756]
frame: 2 [70.72335313629264, 42.45427948015288, 93.20678849602302, 72.32945901253103]
[70.72335313629264, 42.45427948015288, 93.20678849602302, 72.32945901253103]
frame: 3 [71.44596422895413, 40.18087777316183, 92.52283250902349, 66.50209050907307]
[71.44596422895413, 40.18087777316183, 92.52283250902349, 66.50209050907307]
frame: 4 [72.0819581102108, 38.194360139916625, 91.982948297391, 61.4136520977397]
[72.0819581102108, 38.194360139916625, 91.982948297391, 61.4136520977397]
frame: 5 [72.70461372655708, 36.44987292098039, 91.53922051077595, 57.00444798650577]
[72.70461372655708, 36.44987292098039, 91.53922051077595, 57.00444798650577]
frame: 6 [73.25235647347078, 34.8670774288637, 91.09012486983683, 53.06942336212048]
[73.25235647347078, 34.8670774288637, 91.09012486983683, 53.06942336212048]
frame: 7 [73.7997159742809, 33.39063802152363, 90.64984497727374, 49.52346064112334]
[73.7997159742809, 33.39063802152363, 90.64984497727374, 49.52346064112334]
frame: 8 [74.33964891548749, 32.07947825142874, 90.19446878277232, 46.32401496608712]
[74.33964891548749, 32.07947825142874, 90.19446878277232, 46.32401496608712]
frame: 9 [74.80704545543891, 30.894863398402784, 89.74317647908586, 43.44246254765723]
[74.80704545543891, 30.894863398402784, 89.74317647908586, 43.44246254765723]
frame: 10 [75.23365582047252, 29.825739531553893, 89.33571490345336, 40.90703973116146]
[75.23365582047252, 29.825739531553893, 89.33571490345336, 40.90703973116146]
frame: 11 [75.63508291919808, 28.880748411465042, 88.93379260174657, 38.66743267108161]
[75.63508291919808, 28.880748411465042, 88.93379260174657, 38.66743267108161]
frame: 12 [76.02298592783066, 28.04869420987427, 88.55650949052102, 36.688339074899595]
[76.02298592783066, 28.04869420987427, 88.55650949052102, 36.688339074899595]
frame: 13 [76.39282059174934, 27.309084686451303, 88.18895100018156, 34.930228446314416]
[76.39282059174934, 27.309084686451303, 88.18895100018156, 34.930228446314416]
frame: 14 [76.74394839996826, 26.663149104813172, 87.85063656222403, 33.38392899976636]
[76.74394839996826, 26.663149104813172, 87.85063656222403, 33.38392899976636]
frame: 15 [77.07216133135084, 26.096040907341003, 87.53302146845527, 32.0215889038125]
[77.07216133135084, 26.096040907341003, 87.53302146845527, 32.0215889038125]
frame: 16 [77.386230541801, 25.592030039023115, 87.23948018134384, 30.823323999482568]
[77.386230541801, 25.592030039023115, 87.23948018134384, 30.823323999482568]
frame: 17 [77.68317988020497, 25.147977970279985, 86.96296215161996, 29.765176110268104]
[77.68317988020497, 25.147977970279985, 86.96296215161996, 29.765176110268104]
frame: 18 [77.96767286954294, 24.760622054184363, 86.70353015439692, 28.83135059953726]
[77.96767286954294, 24.760622054184363, 86.70353015439692, 28.83135059953726]
frame: 19 [78.2374775743712, 24.416588343537338, 86.45592561789685, 28.007621194959516]
[78.2374775743712, 24.416588343537338, 86.45592561789685, 28.007621194959516]
frame: 20 [78.48802745934606, 24.114571811008506, 86.22119925553922, 27.28143483651797]
[78.48802745934606, 24.114571811008506, 86.22119925553922, 27.28143483651797]
frame: 21 [78.72421819885221, 23.851223058567243, 86.00152334882918, 26.64284184874425]
[78.72421819885221, 23.851223058567243, 86.00152334882918, 26.64284184874425]
frame: 22 [78.94970312784183, 23.616645286118416, 85.79674246102009, 26.07970291158498]
[78.94970312784183, 23.616645286118416, 85.79674246102009, 26.07970291158498]
frame: 23 [79.15962786768203, 23.409738777281277, 85.60566052336529, 25.583585537900294]
[79.15962786768203, 23.409738777281277, 85.60566052336529, 25.583585537900294]
frame: 24 [79.35937892253877, 23.22762385592658, 85.42669871139493, 25.146010751028438]
[79.35937892253877, 23.22762385592658, 85.42669871139493, 25.146010751028438]
frame: 25 [79.54899423843463, 23.065746749931, 85.25849664262529, 24.758920644553815]
[79.54899423843463, 23.065746749931, 85.25849664262529, 24.758920644553815]
frame: 26 [79.7236723581475, 22.923202948105345, 85.1003472269544, 24.416502399705664]
[79.7236723581475, 22.923202948105345, 85.1003472269544, 24.416502399705664]
frame: 27 [79.89399317493114, 22.797559131379458, 84.94873224929889, 24.113369222208924]
[79.89399317493114, 22.797559131379458, 84.94873224929889, 24.113369222208924]
frame: 28 [80.06665600775062, 22.687128189394073, 84.81498058002798, 23.848191262701434]
[80.06665600775062, 22.687128189394073, 84.81498058002798, 23.848191262701434]
frame: 29 [80.23123729815418, 22.590855797908404, 84.68723050483973, 23.61456645982426]
[80.23123729815418, 22.590855797908404, 84.68723050483973, 23.61456645982426]
frame: 30 [80.38573255664596, 22.506789818911205, 84.57033950654784, 23.410034145371007]
[80.38573255664596, 22.506789818911205, 84.57033950654784, 23.410034145371007]
frame: 31 [80.5218327411521, 22.434465890459414, 84.4600807250996, 23.23291744222767]
[80.5218327411521, 22.434465890459414, 84.4600807250996, 23.23291744222767]
frame: 32 [80.64565831005498, 22.369563293879104, 84.35613235680012, 23.07597268367757]
[80.64565831005498, 22.369563293879104, 84.35613235680012, 23.07597268367757]
frame: 33 [80.75870367742401, 22.311938800636874, 84.25630334716838, 22.93665367210166]
[80.75870367742401, 22.311938800636874, 84.25630334716838, 22.93665367210166]
frame: 34 [80.8659812389454, 22.262185776192485, 84.15678325158495, 22.813818156579764]
[80.8659812389454, 22.262185776192485, 84.15678325158495, 22.813818156579764]
frame: 35 [80.95911590992418, 22.218244484276283, 84.05661732407158, 22.70510316150855]
[80.95911590992418, 22.218244484276283, 84.05661732407158, 22.70510316150855]
frame: 36 [81.05086601272814, 22.17861290148088, 83.97064220051529, 22.60939950009209]
[81.05086601272814, 22.17861290148088, 83.97064220051529, 22.60939950009209]
frame: 37 [81.13862519184306, 22.143507996368758, 83.89177994640234, 22.524893682217925]
[81.13862519184306, 22.143507996368758, 83.89177994640234, 22.524893682217925]
frame: 38 [81.21971696736883, 22.112401465900792, 83.81743946579138, 22.450173329689903]
[81.21971696736883, 22.112401465900792, 83.81743946579138, 22.450173329689903]
frame: 39 [81.29651923607176, 22.084809133792607, 83.74765092877834, 22.38400236137473]
[81.29651923607176, 22.084809133792607, 83.74765092877834, 22.38400236137473]
frame: 40 [81.36936904024294, 22.060016487057496, 83.68271358699286, 22.324960707705614]
[81.36936904024294, 22.060016487057496, 83.68271358699286, 22.324960707705614]
frame: 41 [81.44074029348299, 22.038069789251153, 83.62340303218613, 22.272738047157365]
[81.44074029348299, 22.038069789251153, 83.62340303218613, 22.272738047157365]
frame: 42 [81.50821557565716, 22.018658692662488, 83.56703705347559, 22.226560267051987]
[81.50821557565716, 22.018658692662488, 83.56703705347559, 22.226560267051987]
frame: 43 [81.57038371919289, 22.001486975539407, 83.5132089237723, 22.185696625997284]
[81.57038371919289, 22.001486975539407, 83.5132089237723, 22.185696625997284]
frame: 44 [81.6285503023113, 21.98621364654747, 83.46244238136993, 22.14940888847398]
[81.6285503023113, 21.98621364654747, 83.46244238136993, 22.14940888847398]
frame: 45 [81.68421155751, 21.97273800827213, 83.41474840309357, 22.11729447270698]
[81.68421155751, 21.97273800827213, 83.41474840309357, 22.11729447270698]
frame: 46 [81.73556295155772, 21.960748081612895, 83.36917465460701, 22.08882916410996]
[81.73556295155772, 21.960748081612895, 83.36917465460701, 22.08882916410996]
frame: 47 [81.78457982454557, 21.950148231665906, 83.32665152675034, 22.06361473307373]
[81.78457982454557, 21.950148231665906, 83.32665152675034, 22.06361473307373]
frame: 48 [81.83296773319475, 21.94068234164813, 83.28685553566382, 22.04111915426526]
[81.83296773319475, 21.94068234164813, 83.28685553566382, 22.04111915426526]
frame: 49 [81.87906151982254, 21.93231684342518, 83.2502234669888, 22.021251375213243]
[81.87906151982254, 21.93231684342518, 83.2502234669888, 22.021251375213243]
frame: 50 [81.92280018100665, 21.92490965608685, 83.2152249910795, 22.003589902347866]
[81.92280018100665, 21.92490965608685, 83.2152249910795, 22.003589902347866]
frame: 51 [81.96434503062042, 21.918329896427714, 83.18234014229698, 21.987950695836513]
[81.96434503062042, 21.918329896427714, 83.18234014229698, 21.987950695836513]
frame: 52 [82.00338853170223, 21.91253297287023, 83.15160232773344, 21.974120641239207]
[82.00338853170223, 21.91253297287023, 83.15160232773344, 21.974120641239207]
frame: 53 [82.0400352030239, 21.90740114404165, 83.12223352464179, 21.96188342626112]
[82.0400352030239, 21.90740114404165, 83.12223352464179, 21.96188342626112]
Traceback (most recent call last):
  File "test.py", line 128, in <module>
    tester.test()
  File "test.py", line 114, in test
    sample = self[i]
  File "test.py", line 78, in __getitem__
    sample = self._get_sample(idx)
  File "test.py", line 87, in _get_sample
    prev_img = self.transform_prev({'image': prev, 'bb': prevbb})['image']
  File ....../tensorflow/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 49, in __call__
    img = t(img)
  File "....../src/helper.py", line 27, in __call__
    h, w = image.shape[:2]
ValueError: not enough values to unpack (expected 2, got 0)

For some reasons, I replaced my truth path with some dots in the stack trace.

fzh0917 commented 5 years ago

From the information in stack trace, you can see that the bounding box became smaller and smaller instead of out of the ranges.

amoudgl commented 5 years ago

Interesting. Two questions:

I'll try to reproduce the error.

fzh0917 commented 5 years ago

Two answers:

fzh0917 commented 5 years ago

Here is the related code piece in train.py:

parser.add_argument('-n', '--num-batches', default=50, type=int,
                    help='number of total batches to run')
amoudgl commented 5 years ago

Thanks! I feel crop operations in src/helper.py need updates to handle such exceptions (just like original GOTURN). I didn't encounter this issue because I tested the final model. I'll work on it and get back as soon as I have some reasonable results.

fzh0917 commented 5 years ago

OK. Thank you for your timely and detailed responses.