search

zju3dv / EasyMocap

Make human motion capture easier.
Other
3.69k stars 456 forks source link

Trying to run mono demo model with own data #367

Closed thefudgeishot closed 1 year ago

thefudgeishot commented 1 year ago

Trying to run the mono model with command emc --data config/datasets/svimage.yml --exp config/1v1p/hrnet_pare_finetune.yml --root data/video --subs video from the docs

And I'm getting the following error

...
YOLOv5m summary: 290 layers, 21172173 parameters, 0 gradients
Adding AutoShape...
[MultiStage] loading module keypoints2d
[MultiStage] loading module vis2d
[MultiStage] loading module infer
[Run] dataset has 120 samples
[Run]:   0%|                                                                                                               
[Stages] Error in detect
[Run]:   0%|                                                                                                               
Traceback (most recent call last):
  File "/home/user/EasyMocap/myeasymocap/stages/basestage.py", line 77, in at_step
    output = model(**inputs)
  File "/home/user/EasyMocap/myeasymocap/backbone/yolo/yolo.py", line 109, in __call__
    select, res = self.select_class(res, self.name)
  File "/home/user/EasyMocap/myeasymocap/backbone/yolo/yolo.py", line 90, in select_class
    select = np.stack(select)
  File "/home/user/.conda/envs/easymocap/lib/python3.9/site-packages/numpy/core/shape_base.py", line 445, in stack
    raise ValueError('need at least one array to stack')
ValueError: need at least one array to stack

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.conda/envs/easymocap/bin/emc", line 33, in <module>
    sys.exit(load_entry_point('easymocap', 'console_scripts', 'emc')())
  File "/home/user/EasyMocap/apps/mocap/run.py", line 99, in main_entrypoint
    process(dataset, model)
  File "/home/user/EasyMocap/apps/mocap/run.py", line 11, in process
    ret = model.at_step(data, i)
  File "/home/user/EasyMocap/myeasymocap/stages/basestage.py", line 80, in at_step
    raise Exception
Exception

Any idea what may be causing this, I have doubled checked that all models have been installed and are in the correct directory. I should also note that after downloading the demo data the command executed perfectly and produced the correct outputs.

thefudgeishot commented 1 year ago

My annots files look similar to this, generated with YOLOv5+HRNet 


{
    "filename": "images/ollie/000063.jpg",
    "height": 1080,
    "width": 1920,
    "annots": [
        {
            "personID": 0,
            "bbox": [832.35, 166.83, 1265.88, 573.76, 1.00],
            "keypoints": [
              [1039.71,  259.27,    0.96], 
              [1067.93,  249.87,    0.93], 
              [1017.13,  240.46,    0.93], 
              [ 941.86,  244.22,    0.90], 
              [ 855.30,  274.33,    0.90], 
              [1118.74,  259.27,    0.93], 
              [1186.48,  300.67,    0.92], 
              [1227.87,  345.83,    0.96], 
              [1060.40,  349.59,    0.89], 
              [1028.42,  342.07,    0.90], 
              [ 983.26,  330.78,    0.96], 
              [ 964.44,  458.73,    0.92], 
              [1092.39,  357.12,    0.89], 
              [1133.79,  421.10,    1.01], 
              [1103.68,  541.52,    0.93], 
              [1032.18,  247.98,    0.99], 
              [1051.00,  251.75,    0.98], 
              [1024.65,  232.93,    0.93], 
              [1077.34,  232.93,    0.96], 
              [   0.00,    0.00,    0.00], 
              [   0.00,    0.00,    0.00], 
              [   0.00,    0.00,    0.00], 
              [   0.00,    0.00,    0.00], 
              [   0.00,    0.00,    0.00], 
              [   0.00,    0.00,    0.00]
            ],
            "isKeyframe": false
        }
    ],
    "isKeyframe": false
}
chingswy commented 1 year ago

Hello, this script will automaticly run yolo. Does this image contain no person?

thefudgeishot commented 1 year ago

Thanks for the swift response, the first frames do contain a person. Could I quickly ask if there is an argument to pass into the command to ignore invalid frames? Also is the model capable of mo-capping a person if they are half in frame, or are they required to be completely in frame for the duration of the video. Thanks again.

raflitp commented 5 months ago

Have you figured out this problem ? Especially when someone is not in frame or half body in frame ??

CHDTevior commented 1 month ago

I checked the problem is because the yolo does not return a correct bbox, so maybe you can edit yolo.py to let it return the default bbox if no object is detected as placeholder.

chen-hao-ran commented 3 weeks ago

@thefudgeishot Hi, I'm struggling to use my own dataset, due to the demo dataset is not available now. I have no idea of what the structure of my own dataset should be like. I'll appreciate it if you could share your downloaded demo or dataset structure. I'm looking forward to your early reply.

CHDTevior commented 3 weeks ago

@thefudgeishot Hi, I'm struggling to use my own dataset, due to the demo dataset is not available now. I have no idea of what the structure of my own dataset should be like. I'll appreciate it if you could share your downloaded demo or dataset structure. I'm looking forward to your early reply.

Are you trying to use your own monocular video?

chen-hao-ran commented 3 weeks ago

@thefudgeishot Hi, I'm struggling to use my own dataset, due to the demo dataset is not available now. I have no idea of what the structure of my own dataset should be like. I'll appreciate it if you could share your downloaded demo or dataset structure. I'm looking forward to your early reply.

Are you trying to use your own monocular video? @CHDTevior Thanks for your swift reply. Yes, and I'm struggling with the dataset structure.

CHDTevior commented 3 weeks ago

@thefudgeishot Hi, I'm struggling to use my own dataset, due to the demo dataset is not available now. I have no idea of what the structure of my own dataset should be like. I'll appreciate it if you could share your downloaded demo or dataset structure. I'm looking forward to your early reply.

Are you trying to use your own monocular video? @CHDTevior Thanks for your swift reply. Yes, and I'm struggling with the dataset structure.

Can you give me your wechat or email address so I can send it to u.

CHDTevior commented 3 weeks ago

internet-rotate (1).zip or i just put it here

chen-hao-ran commented 3 weeks ago

internet-rotate (1).zip or i just put it here

Thank you so much!! yeah, it's my wechat: a467631698 and I'd hope to discuss some topics related to 3D vision with you

chen-hao-ran commented 3 weeks ago

internet-rotate (1).zip or i just put it here

@CHDTevior Hi, there is another problem. how can i get camera info? It seems like there is no camera info related file in the output dir

CHDTevior commented 3 weeks ago

I can’t find u on WeChat, mine is 2458783152

---- Replied Message ---- | From | @.> | | Date | 10/12/2024 03:42 | | To | zju3dv/EasyMocap @.> | | Cc | CHDTevior @.>, Mention @.> | | Subject | Re: [zju3dv/EasyMocap] Trying to run mono demo model with own data (Issue #367) |

internet-rotate (1).zip or i just put it here

Thank you so much!! yeah, it's my wechat: a467631698 and I'd hope to discuss some topics related to 3D vision with you

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

CHDTevior commented 3 weeks ago

They set mono camera into a pre-defined position.

---- Replied Message ---- | From | @.> | | Date | 10/12/2024 04:48 | | To | zju3dv/EasyMocap @.> | | Cc | CHDTevior @.>, Mention @.> | | Subject | Re: [zju3dv/EasyMocap] Trying to run mono demo model with own data (Issue #367) |

internet-rotate (1).zip or i just put it here

@CHDTevior Hi, there is another problem. how can i get camera info? It seems like there is no camera info related file in the output dir

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

xianrui-luo commented 2 weeks ago

They set mono camera into a pre-defined position. … ---- Replied Message ---- | From | @.> | | Date | 10/12/2024 04:48 | | To | zju3dv/EasyMocap @.> | | Cc | CHDTevior @.>, Mention @.> | | Subject | Re: [zju3dv/EasyMocap] Trying to run mono demo model with own data (Issue #367) | internet-rotate (1).zip or i just put it here @CHDTevior Hi, there is another problem. how can i get camera info? It seems like there is no camera info related file in the output dir — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

Hi how do I get the pre-defined extri and intri of the camera?

CHDTevior commented 2 weeks ago

They set mono camera into a pre-defined position. … ---- Replied Message ---- | From | @.**> | | Date | 10/12/2024 04:48 | | To | zju3dv/EasyMocap @.**> | | Cc | CHDTevior @.**>, Mention @.**> | | Subject | Re: [zju3dv/EasyMocap] Trying to run mono demo model with own data (Issue #367) | internet-rotate (1).zip or i just put it here @CHDTevior Hi, there is another problem. how can i get camera info? It seems like there is no camera info related file in the output dir — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

Hi how do I get the pre-defined extri and intri of the camera?

Need to do nothing.

xianrui-luo commented 2 weeks ago

They set mono camera into a pre-defined position. … ---- Replied Message ---- | From | @.**> | | Date | 10/12/2024 04:48 | | To | zju3dv/EasyMocap @.**> | | Cc | CHDTevior @.**>, Mention @.**> | | Subject | Re: [zju3dv/EasyMocap] Trying to run mono demo model with own data (Issue #367) | internet-rotate (1).zip or i just put it here @CHDTevior Hi, there is another problem. how can i get camera info? It seems like there is no camera info related file in the output dir — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

Hi how do I get the pre-defined extri and intri of the camera?

Need to do nothing.

I use the pre-defined camera for training, image

but the smpl does not align. I think the camera parameters setting is incorrect image

CHDTevior commented 2 weeks ago

the pre-defined camera, in my memory, has been set as center of world coordinate

xianrui-luo commented 2 weeks ago

Thank you I get the world-to-camera transform and now it is ok

CHDTevior commented 2 weeks ago

Thank you I get the world-to-camera transform and now it is ok

Any further problems we can discuss on Wchat.