search

DirtyHarryLYL / HAKE-Action

As a part of the HAKE project, includes the reproduced SOTA models and the corresponding HAKE-enhanced versions (CVPR2020).
Apache License 2.0
100 stars 13 forks source link

Instance-level HOI Demo #4

Closed chentairan closed 4 years ago

chentairan commented 4 years ago

Hi,

It seems that this code does not include the part of human part or object detection. I don't understand why iCAN is used as the object detection? iCAN seems to use Faster-RCNN for object detection.

Is there any way to input an image and output the corresponding instance-level human interaction behavior as a demo?

Thanks, Tairan Chen

DirtyHarryLYL commented 4 years ago

We adopt the two-stage HOI detection pipeline, i.e., first use Faster R-CNN to detect the human/object and estimate human pose. Then we use these detections for the next stage learning.

iCAN has provided their detected boxes, we use their detected boxes and run Alphapose to obtain the corresponding human poses. The part box generation process is here: https://github.com/DirtyHarryLYL/HAKE-Action/blob/Instance-level-HAKE-Action/script/part_box_generation.py.

The essential inputs include: human and object boxes, human pose. Thus, you need to set up the Faster R-CNN and Alphapose to run in one-shot.

DirtyHarryLYL commented 4 years ago

Thanks for your advice, we may consider adding a demo to include the whole processing.

DirtyHarryLYL commented 4 years ago

You could also refer to the iCAN demo: https://github.com/vt-vl-lab/iCAN#demotest-on-your-own-images, to detect the boxes. Then additionally running pose estimation and generating part boxes.

chentairan commented 4 years ago

Thank you very much for your reply.

Do you mean that you use the faster-rcnn weight trained by iCAN for object detection to get the bbox of COCO or HICO-DET dataset.

DirtyHarryLYL commented 4 years ago

Yeah, the detection boxes from iCAN are on their website: http://chengao.vision/iCAN/. Generally, HOI detection methods often use the COCO pre-trained faster rcnn to detect the human and object.

chentairan commented 4 years ago

I see, thx. I will make a demo for this.

chentairan commented 4 years ago

Hi, Which version of tensorflow did you use to build the code ? And it seems iCAN is based on python2.7 and tensorflow1.1.0 or 1.2.0, did you test the code on python3 and a higher version of tensorflow like 1.4.1 ~ 1.15.0 ?

Thank, Tairan

DirtyHarryLYL commented 4 years ago

We use py2.7 and tf 1.10.0. The other versions are not tested on our model now. More details about the requirement are here: https://github.com/DirtyHarryLYL/HAKE-Action/blob/Instance-level-HAKE-Action/requirements.txt.

chentairan commented 4 years ago

Thx！Looking forward to the quick demo.