Inference code

[abhigoku10]

@fredzzhang @nikanor97 hi thanks for sharing the code base great work

1. do we inference code which we can run on the custom data / custom videos and visualize the results
2. currently when i tested the model using test.py for some scenes present in the validation data like an only person running on a beach without any other object present there is no detections/activity in the output, is there any way i can get results like people walking , fighting, waving without depending on the object present in the scene

Thanks in advance

[fredzzhang]

Hi, @abhigoku10

Thanks for taking interest in our work.

do we inference code which we can run on the custom data / custom videos and visualize the results

You can try to set up your customised dataset in the same way as HICODet here

is there any way i can get results like people walking , fighting, waving without depending on the object present in the scene

Unfortunately no. The model is designed for human-object interaction, which involves a (human, verb, object) triplet.

[kruthikakr]

Hi seeking help I am trying to run the code on the image from the dataset. where can i get Inference script to test the image in the HICODet dataset ?

Thank you

[fredzzhang]

Hi @kruthikakr

You can find demo.py under the directory diagnosis.

[kruthikakr]

@fredzzhang Thank you .. i was able to run the demo.

But for the custom data , can you give some more inputs for data preparation so that the model can be used to find the human object interaction.

[fredzzhang]

Hi @kruthikakr,

Please refer to fredzzhang/hicodet#18 and #63.

Fred.

[afzaalhussain50]

Hi @fredzzhang, i am using demo.py for inference, but getting an error invalid keyError output['labels'] in visualize_entire_image() function because you are passing output[0] of network to visualize_entire_image(dataset, output[0]) but net output does not contains ['labels']

from where i can get these labels, either these are detections labels or something other which i am missing?

Thanks

[fredzzhang]

Hi @afzaalhussain50,

The labels are computed based on the ground truth detections. See line 128 in demo.py,

...
output = net(image, detection, target)
...

The target is passed to the network as well. You probably changed the code somewhere.

Fred.

[afzaalhussain50]

@fredzzhang I am using demo.py on custom dataset which is not labeled so i am passing target as None, which you are also mentioned in an other issue. https://github.com/fredzzhang/spatially-conditioned-graphs/issues/63#issuecomment-917925198

detection = a dictionary with {'boxes', 'labels' and 'scores'} which are computed by preprocessing.py output = net(image, detection, None) and output[0] have these dictionary items where 'labels' is not present, output[0] shown in image.

and also guide me one thing more how i can compute hoi (human action object) triplet using above network results?

Thanks for your quick response.

[fredzzhang]

Hi @afzaalhussain50,

If you are using custom dataset, just comment out the lines that referenced labels. That'd be line 71 and 82.

Fred.