liruilong940607 / Pose2Seg

Code for the paper "Pose2Seg: Detection Free Human Instance Segmentation" @ CVPR2019.
http://www.liruilong.cn/projects/pose2seg/index.html
MIT License
529 stars 136 forks source link

How to run it for our own set of images/video ? #12

Open manansaxena opened 5 years ago

manansaxena commented 5 years ago

Hi, I love the work you have done. But I wanted to run it on my own set of images or video ? How should I do that?

liruilong940607 commented 5 years ago

Thanks for your interests.

If you don't want to go over the code, the fastest way of doing it is to generate a json file with the same format as the coco data json file, and use the test.py to test it.

Note that our method takes both image and keypoints as the input, so human keypoints should be contained in the json file. You may need to first detect the person keypoints using some other methods, such as the github repos [openpose], [pose-ae-train], [alphapose] etc.

manansaxena commented 5 years ago

Thanks for the reply One more thing - I have three people in the video and I want to create a binary mask i.e. at one time only one person can be seen in the video. So where should I make changes in the code to make this possible. Thanks

liruilong940607 commented 5 years ago

I think you need a video post-processing process after you get the segmentation mask.

You can easily implement this using some opencv functions.

wine3603 commented 5 years ago

Thanks for the reply One more thing - I have three people in the video and I want to create a binary mask i.e. at one time only one person can be seen in the video. So where should I make changes in the code to make this possible. Thanks

Do you want to track the human objects in the video? If so I think you need to add some tracking tips. Human poses segmentation maybe different frame to frame. I am interested in this topic, too.

marteiro commented 5 years ago

This is amazing work. I'm trying to sun the test, investigate the result and later try to apply this to a live video feed. Right now I was able to run test.py with the paper's final weight but I could't seen any result files.

Screenshot from 2019-07-04 09-10-57

It this right? How can I see the results?

Thanks ,

manansaxena commented 5 years ago

Thanks for the reply One more thing - I have three people in the video and I want to create a binary mask i.e. at one time only one person can be seen in the video. So where should I make changes in the code to make this possible. Thanks

Do you want to track the human objects in the video? If so I think you need to add some tracking tips. Human poses segmentation maybe different frame to frame. I am interested in this topic, too.

Hey, Were you able to solve the problem of converting openpose keypoints to coco format ?

wine3603 commented 5 years ago

Thanks for the reply One more thing - I have three people in the video and I want to create a binary mask i.e. at one time only one person can be seen in the video. So where should I make changes in the code to make this possible. Thanks

Do you want to track the human objects in the video? If so I think you need to add some tracking tips. Human poses segmentation maybe different frame to frame. I am interested in this topic, too.

Hey, Were you able to solve the problem of converting openpose keypoints to coco format ?

I use openpose to estimate the joints and output in coco json format, then I made a new test data set in the form of OCHuman to run the test, but there are too many bugs. Does anyone succeed to run on your only image?

manansaxena commented 5 years ago

No I have done the same thing but new bugs just keep coming up.

manansaxena commented 5 years ago

@liruilong940607 Hi, I formed a json file with coco format. I have also arranged the data just like in the original documentation. But when I run test.py it gets an error that - x1, y1, bboxw, bboxh = obj['bbox'] ValueError: not enough values to unpack (expected 4, got 0)

You said to leave these fields blank in some other issue: 'area', 'iscrowd', 'segmentation', 'bbox', because we don't use them for inference. 1)What did you mean by blank ? 2)I only need inference(just the segmented output image and masks), so for this what changes are needed to be done in test.py

Thanks

ManiaaJia commented 5 years ago

@liruilong940607 @manansaxena @wine3603 @marteiro Hello everyone, I am sorry to bother you.But I can't access the dataset address provided by the dataset repo. Could you download it for me or give me a link to download? Thanks a lot.

azuic commented 5 years ago

@manansaxena hi! Do you mind sharing your final input data format? I got problems with category_id, which returns me a KeyError: 1 both when I set category_id to 1 in the input annotation file and when I remove this key from the file. Should I set the value to None instead? Thanks!

azuic commented 5 years ago

I fixed my issue above, but I am getting the same 'bbox' warning as you have, do you mind sharing how you solved this problem?

manansaxena commented 5 years ago

I made a lot of changes in Cocodatasetinfo.py, remove all the bbox code from it. But then again I got very poor results on in the wild images, I can't say for sure that it isn't the problem, even though according to paper for inference it shouldn't be required

pixelite1201 commented 4 years ago

Did anyone manage to run it on their own set of images?

ziBLan commented 4 years ago

Did anyone manage to run it on their own set of images?

I am trying ....