Demo failed to infer the image demo/demo.jpg

[meassinal]

Hi,

First of all, I'd like to thank you for the code.

I was trying to experiment the network and I had no luck!

I used pre-trained model occlusion_net.pth in order to experiment the demo but it failed. I've tried to debug to find the cause of the problem but I was not lucky. Below is the printed message:

Using MLP graph encoder.
Using learned graph decoder.
Using MLP graph encoder.
Prepairing to visualize the rusult...
car
Before coco demo overlay keypoints graph
Drawing keypoints...
[[0, 2], [1, 3], [0, 1], [2, 3], [9, 11], [10, 12], [9, 10], [11, 12], [4, 0], [4, 9], [4, 5], [5, 1], [5, 10], [6, 2], [6, 11], [7, 3], [7, 12], [6, 7]]
[[1528.4391  1647.2606  1633.7855  1776.1908  1526.5496  1594.0615
  1702.2465  1801.6003  1808.0066  1585.1688  1684.1361  1720.7932
  1773.1305  1498.484  ]
 [ 370.9784   370.58386  365.24368  362.67     315.4024   323.2245
   301.12967  293.40402  233.80951  252.78555  249.06302  239.7588
   242.87587  387.13684]
 [   0.       100.         0.       100.       100.       100.
     0.       100.         0.       100.       100.         0.
   100.         0.     ]]
<class 'numpy.ndarray'>
0 2 (1528.4391, 370.9784) (1633.7855, 365.24368)
Fail to infer for image demo/demo.jpg. Skipped.
2021-10-23_05-14-38
2021-10-23_05-14-39

[dineshreddy91]

do you mean to say the demo results does not visualize the image?. Are you using the correct docker image?

[meassinal]

I think I used the correct docker image as I could get the json result but not a visualized one in image. Apparently, the demo could not visualize the image. I traced the error and it happened in lib/predictor.py in def vis_keypoints_car_rainbow somewhere around drawing the keypoints.

[dineshreddy91]

can you remove the try catch in https://github.com/dineshreddy91/Occlusion_Net/blob/ce60b6b88ee02de33924aa9363bcba1a9c7f75e5/infer.py and let me know the error?

[meassinal]

Here is the error message after removing try catch. It is because of datatype of points passed to cv2.line, which integer should be passed instead.

Traceback (most recent call last):
  File "infer.py", line 151, in <module>
    main()
  File "infer.py", line 135, in main
    result = coco_demo.overlay_keypoints_graph(result, top_predictions,vis_color , target='car')
  File "/code/lib/predictor.py", line 327, in overlay_keypoints_graph
    image = vis_keypoints_car_rainbow(image, region.transpose((1, 0)),kp_thresh = -10)
  File "/code/lib/predictor.py", line 565, in vis_keypoints_car_rainbow
    color=colors[l], thickness=2, lineType=cv2.LINE_AA)
cv2.error: OpenCV(4.5.3) :-1: error: (-5:Bad argument) in function 'line'
> Overload resolution failed:
>  - Can't parse 'pt1'. Sequence item with index 0 has a wrong type
>  - Can't parse 'pt1'. Sequence item with index 0 has a wrong type

After casting points to integer, I could get a visualized result image. In lib/predictor.py and def vis_keypoints_car_rainbow I convert the points to integer in the following:

p1 = int(kps[0, i1]), int(kps[1, i1])
p2 = int(kps[0, i2]), int(kps[1, i2])

Also, here is the demo result but I wonder why the car in the front was not detected and was not drawn with keypoints. Or is it because I only use dataset car_craig1 ?

[dineshreddy91]

Thanks for pointing out the issue. I have updated the code to fix this issue. Using one sequence from dataset might be causing this issue. please train on all the sequences for better detection accuracy.

[meassinal]

Thank you so much for your kind support. I will try and let you know the result :)

[meassinal]

I'm thinking of experimenting it real time in browser by converting the Pytorch model to js via ONNX. Could you advice if it's possible? Thanks in advance.

[zapaishchykova]

Hello @dineshreddy91, I'm having the same issue when running test on demo image

sh test.sh occlusion_net demo/demo.jpg
Using MLP graph encoder.
Using learned graph decoder.
Using MLP graph encoder.
Fail to infer for image demo/demo.jpg. Skipped.
2021-11-09_10-01-36
2021-11-09_10-01-36

when removing try catch i get:

Using MLP graph encoder.
Using learned graph decoder.
Using MLP graph encoder.
Traceback (most recent call last):
  File "infer.py", line 149, in <module>
    main()
  File "infer.py", line 133, in main
    result = coco_demo.overlay_keypoints_graph(result, top_predictions,vis_color , target='car')
  File "/code/lib/predictor.py", line 327, in overlay_keypoints_graph
    image = vis_keypoints_car_rainbow(image, region.transpose((1, 0)),kp_thresh = -10)
  File "/code/lib/predictor.py", line 560, in vis_keypoints_car_rainbow
    color=colors[l], thickness=2, lineType=cv2.LINE_AA)
cv2.error: OpenCV(4.5.4-dev) :-1: error: (-5:Bad argument) in function 'line'
> Overload resolution failed:
>  - Can't parse 'pt1'. Sequence item with index 0 has a wrong type
>  - Can't parse 'pt1'. Sequence item with index 0 has a wrong type

looks like its the same error as was described above, could you please update the repo? many thanks!

[dineshreddy91]

This issue has been updated in the repository. Can you try git pull with the new repo and try again.

[lchiaying]

Hi! I pulled your repo today and encountered the same issue as the two commenters above. I think it's due to an incorrect dtype that is expected by cv2.line. It worked when I changed lines 555-556 in lib/predictor.py to:

p1 = np.int32([kps[0, i1], kps[1, i1]])
p2 = np.int32([kps[0, i2], kps[1, i2]])

Many thanks for sharing the code!