open-mmlab / mmaction2

OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark
https://mmaction2.readthedocs.io
Apache License 2.0
4.27k stars 1.24k forks source link

Run "demo/webcam_demo_spatiotemporal_det.py" , the program is stuck. #1199

Open Z-Xiong opened 3 years ago

Z-Xiong commented 3 years ago

When I run "demo/webcam_demo_spatiotemporal_det.py" with USB cam or RTSP web cam , the program is stuck after running for a period of time.

I think this is caused by the inference speed or the display speed being slower than the reading speed.

Some log information is as follows:(Sometimes directly show killed)

INFO:__main__:Stdet Results: [[('sit', 0.8451391), ('carry/hold (an object)', 0.52010876)]]
DEBUG:__main__:Main thread inference time 627 ms
DEBUG:__main__:Read thread: 691 ms, 12 fps
DEBUG:__main__:Read thread: 231 ms, 35 fps
INFO:__main__:Stdet Results: [[('sit', 0.84119666), ('carry/hold (an object)', 0.5008541)]]
DEBUG:__main__:Main thread inference time 600 ms
DEBUG:__main__:Read thread: 296 ms, 27 fps
+++++++++++++++++++++++++++++++++++++++++++++++++++++++

DEBUG:__main__:Read thread: 212 ms, 38 fps
DEBUG:__main__:Read thread: 332 ms, 24 fps
DEBUG:__main__:Read thread: 350 ms, 23 fps
+++++++++++++++++++++++++++++++++++++++++++++++++++++++

INFO:__main__:Stdet Results: [[('sit', 0.8407789), ('carry/hold (an object)', 0.48366198)]]
DEBUG:__main__:Main thread inference time 1010 ms
DEBUG:__main__:Read thread: 199 ms, 40 fps
+++++++++++++++++++++++++++++++++++++++++++++++++++++++

DEBUG:__main__:Read thread: 320 ms, 25 fps
INFO:__main__:Stdet Results: [[('sit', 0.8350647), ('carry/hold (an object)', 0.46698186), ('text on/look at a cellphone', 0.42576995)]]
DEBUG:__main__:Main thread inference time 657 ms
DEBUG:__main__:Read thread: 250 ms, 32 fps
DEBUG:__main__:Read thread: 247 ms, 32 fps
+++++++++++++++++++++++++++++++++++++++++++++++++++++++

INFO:__main__:Stdet Results: [[('sit', 0.80609334), ('carry/hold (an object)', 0.48520392), ('text on/look at a cellphone', 0.5238202)]]
DEBUG:__main__:Main thread inference time 382 ms
DEBUG:__main__:Read thread: 251 ms, 32 fps
DEBUG:__main__:Read thread: 316 ms, 25 fps
DEBUG:__main__:Read thread: 273 ms, 29 fps
INFO:__main__:Stdet Results: [[('sit', 0.7361066), ('carry/hold (an object)', 0.47983062)]]
DEBUG:__main__:Main thread inference time 957 ms
+++++++++++++++++++++++++++++++++++++++++++++++++++++++

DEBUG:__main__:Read thread: 256 ms, 31 fps
DEBUG:__main__:Read thread: 243 ms, 33 fps
DEBUG:__main__:Read thread: 300 ms, 27 fps
DEBUG:__main__:Read thread: 273 ms, 29 fps
+++++++++++++++++++++++++++++++++++++++++++++++++++++++

INFO:__main__:Stdet Results: [[('sit', 0.7959219), ('carry/hold (an object)', 0.53366905)]]
DEBUG:__main__:Main thread inference time 970 ms
DEBUG:__main__:Read thread: 208 ms, 39 fps
DEBUG:__main__:Read thread: 350 ms, 23 fps
DEBUG:__main__:Read thread: 238 ms, 34 fps
INFO:__main__:Stdet Results: [[('sit', 0.82904106), ('carry/hold (an object)', 0.60257)]]
DEBUG:__main__:Main thread inference time 813 ms
+++++++++++++++++++++++++++++++++++++++++++++++++++++++

DEBUG:__main__:Read thread: 217 ms, 37 fps
DEBUG:__main__:Read thread: 318 ms, 25 fps
DEBUG:__main__:Read thread: 332 ms, 24 fps
INFO:__main__:Stdet Results: [[('sit', 0.8334102), ('carry/hold (an object)', 0.6356909)]]
DEBUG:__main__:Main thread inference time 1003 ms
+++++++++++++++++++++++++++++++++++++++++++++++++++++++

DEBUG:__main__:Read thread: 312 ms, 26 fps
DEBUG:__main__:Read thread: 228 ms, 35 fps
+++++++++++++++++++++++++++++++++++++++++++++++++++++++

DEBUG:__main__:Read thread: 1157 ms, 7 fps
INFO:__main__:Stdet Results: [[('sit', 0.8563365), ('carry/hold (an object)', 0.6350206)]]
DEBUG:__main__:Main thread inference time 8273 ms
DEBUG:__main__:Read thread: 11494 ms, 1 fps

Process finished with exit code 137 (interrupted by signal 9: SIGKILL)

I have adjust the keyframes:

keyframe = task.frames[len(task.frames) // 4]

However, it will still get stuck.

What parameters should I adjust so that the log information output will not get stuck.

Z-Xiong commented 3 years ago

Parameter setting:

def parse_args():
    parser = argparse.ArgumentParser(
        description='MMAction2 webcam spatio-temporal detection demo')

    parser.add_argument(
        '--config',
        default=('../configs/detection/ava/'
                 'slowonly_omnisource_pretrained_r101_8x8x1_20e_ava_rgb.py'),
        help='spatio temporal detection config file path')
    parser.add_argument(
        '--checkpoint',
        default=('https://download.openmmlab.com/mmaction/detection/ava/'
                 'slowonly_omnisource_pretrained_r101_8x8x1_20e_ava_rgb/'
                 'slowonly_omnisource_pretrained_r101_8x8x1_20e_ava_rgb'
                 '_20201217-16378594.pth'),
        help='spatio temporal detection checkpoint file/url')
    parser.add_argument(
        '--action-score-thr',
        type=float,
        default=0.4,
        help='the threshold of human action score')
    parser.add_argument(
        '--det-config',
        default='../demo/faster_rcnn_r50_fpn_2x_coco.py',
        help='human detection config file path (from mmdet)')
    parser.add_argument(
        '--det-checkpoint',
        default=('http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/'
                 'faster_rcnn_r50_fpn_2x_coco/'
                 'faster_rcnn_r50_fpn_2x_coco_'
                 'bbox_mAP-0.384_20200504_210434-a5d8aa15.pth'),
        help='human detection checkpoint file/url')
    parser.add_argument(
        '--det-score-thr',
        type=float,
        default=0.9,
        help='the threshold of human detection score')
    parser.add_argument(
        '--input-video',
        default='0',
        type=str,
        help='webcam id or input video file/url')
    parser.add_argument(
        '--label-map', default='../demo/label_map_ava.txt', help='label map file')
    parser.add_argument(
        '--device', type=str, default='cuda:0', help='CPU/CUDA device option')
    parser.add_argument(
        '--output-fps',
        default=15,
        type=int,
        help='the fps of demo video output')
    parser.add_argument(
        '--out-filename',
        default=None,
        type=str,
        help='the filename of output video')
    parser.add_argument(
        '--show',
        # action='store_true',
        default=True,
        help='Whether to show results with cv2.imshow')
    parser.add_argument(
        '--display-height',
        type=int,
        default=0,
        help='Image height for human detector and draw frames.')
    parser.add_argument(
        '--display-width',
        type=int,
        default=0,
        help='Image width for human detector and draw frames.')
    parser.add_argument(
        '--predict-stepsize',
        default=8,
        type=int,
        help='give out a prediction per n frames')
    parser.add_argument(
        '--clip-vis-length',
        default=8,
        type=int,
        help='Number of draw frames per clip.')
    parser.add_argument(
        '--cfg-options',
        nargs='+',
        action=DictAction,
        default={},
        help='override some settings in the used config, the key-value pair '
        'in xxx=yyy format will be merged into config file. For example, '
        "'--cfg-options model.backbone.depth=18 model.backbone.with_cp=True'")

    args = parser.parse_args()
    return args
SamShen12345 commented 1 year ago

Hi, Do you solve this problem?

Z-Xiong commented 1 year ago

solve it. adjust param.

xqc-qc commented 1 year ago

solve it. adjust param.

Hello, how is this parameter adjusted? Do you change the parameters according to this? keyframe = task.frames[len(task.frames) // 4] Try changing the number 4, make it bigger?

Z-Xiong commented 1 year ago

solve it. adjust param.

Hello, how is this parameter adjusted? Do you change the parameters according to this? keyframe = task.frames[len(task.frames) // 4] Try changing the number 4, make it bigger?

I can't rememeber it. May adjust "--predict-stepsize --output-fps". No imshow.

xqc-qc commented 1 year ago

solve it. adjust param.

Hello, how is this parameter adjusted? Do you change the parameters according to this? keyframe = task.frames[len(task.frames) // 4] Try changing the number 4, make it bigger?

I can't rememeber it. May adjust "--predict-stepsize --output-fps". No imshow.

OK, thank you. Is your camera used in conjunction with the GUI interface? Is there video behavior detection during camera detection?