Docker won't run

[davidhorrock0949]

Hi,

I can't get the docker image to run, it gives me the following error:

docker run rlleshi/phar python src/demo/multimodial_demo.py video.mp4 demo.json --timestamps

Traceback (most recent call last):
  File "__init__.cython-30.pxd", line 984, in numpy.import_array
RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf . Check the section C-API incompatibility at the Troubleshooting ImportError section at https://numpy.org/devdocs/user/troubleshooting-importerror.html#c-api-incompatibility for indications on how to solve this problem .

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/phar/mmaction2/demo/demo_skeleton.py", line 23, in <module>
    from mmpose.apis import (inference_top_down_pose_model, init_pose_model,
  File "/workspace/phar/mmpose/mmpose/apis/__init__.py", line 2, in <module>
    from .inference import (inference_bottom_up_pose_model,
  File "/workspace/phar/mmpose/mmpose/apis/inference.py", line 14, in <module>
    from mmpose.datasets.dataset_info import DatasetInfo
  File "/workspace/phar/mmpose/mmpose/datasets/__init__.py", line 7, in <module>
    from .datasets import (  # isort:skip
  File "/workspace/phar/mmpose/mmpose/datasets/datasets/__init__.py", line 2, in <module>
    from ...deprecated import (TopDownFreiHandDataset, TopDownOneHand10KDataset,
  File "/workspace/phar/mmpose/mmpose/deprecated.py", line 5, in <module>
    from .datasets.datasets.base import Kpt2dSviewRgbImgTopDownDataset
  File "/workspace/phar/mmpose/mmpose/datasets/datasets/base/__init__.py", line 2, in <module>
    from .kpt_2d_sview_rgb_img_bottom_up_dataset import \
  File "/workspace/phar/mmpose/mmpose/datasets/datasets/base/kpt_2d_sview_rgb_img_bottom_up_dataset.py", line 8, in <module>
    from xtcocotools.coco import COCO
  File "/opt/conda/lib/python3.8/site-packages/xtcocotools/coco.py", line 58, in <module>
    from . import mask as maskUtils
  File "/opt/conda/lib/python3.8/site-packages/xtcocotools/mask.py", line 3, in <module>
    import xtcocotools._mask as _mask
  File "xtcocotools/_mask.pyx", line 23, in init xtcocotools._mask
  File "__init__.cython-30.pxd", line 986, in numpy.import_array
ImportError: numpy.core.multiarray failed to import

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/demo/multimodial_demo.py", line 20, in <module>
    from demo.demo_skeleton import frame_extraction
  File "/workspace/phar/mmaction2/demo/demo_skeleton.py", line 26, in <module>
    raise ImportError('Failed to import `inference_top_down_pose_model`, '
ImportError: Failed to import `inference_top_down_pose_model`, `init_pose_model`, and `vis_pose_result` form `mmpose.apis`. These apis are required in this demo! 

When I compile from source, I get almost the same error:

/ml/phar/mmcv/mmcv/cnn/bricks/transformer.py:28: UserWarning: Fail to import ``MultiScaleDeformableAttention`` from ``mmcv.ops.multi_scale_deform_attn``, You should install ``mmcv-full`` if you need this module. 
  warnings.warn('Fail to import ``MultiScaleDeformableAttention`` from '
Traceback (most recent call last):
  File "/ml/phar/src/demo/demo_skeleton.py", line 15, in <module>
    from mmdet.apis import inference_detector, init_detector
  File "/ml/phar/mmdet/mmdet/apis/__init__.py", line 1, in <module>
    from .inference import (async_inference_detector, inference_detector,
  File "/ml/phar/mmdet/mmdet/apis/inference.py", line 6, in <module>
    from mmcv.ops import RoIPool
  File "/ml/phar/mmcv/mmcv/ops/__init__.py", line 2, in <module>
    from .assign_score_withk import assign_score_withk
  File "/ml/phar/mmcv/mmcv/ops/assign_score_withk.py", line 5, in <module>
    ext_module = ext_loader.load_ext(
  File "/ml/phar/mmcv/mmcv/utils/ext_loader.py", line 13, in load_ext
    ext = importlib.import_module('mmcv.' + name)
  File "/home/firebug/miniconda3/envs/nsfw2/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'mmcv._ext'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/ml/phar/src/demo/demo_skeleton.py", line 17, in <module>
    raise ImportError('Failed to import `inference_detector` and '
ImportError: Failed to import `inference_detector` and `init_detector` form `mmdet.apis`. These apis are required in this demo! 

When compiling from source, I made sure I had the right versions of the libraries by compiling from the exact git checkpoints referenced by your code:

mmaction2 0.23.0 /ml/phar/mmaction2 mmcv 1.3.18 /ml/phar/mmcv mmdet 2.12.0 /ml/phar/mmdet mmpose 0.22.0 /ml/phar/mmpose

Any ideas? Thanks.

[AshoPash]

I tried using docker and i am getting the same error ( more or less ) :

No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
Traceback (most recent call last):
  File "/workspace/phar/mmaction2/demo/demo_skeleton.py", line 16, in <module>
    from mmdet.apis import inference_detector, init_detector
  File "/opt/conda/lib/python3.8/site-packages/mmdet/apis/__init__.py", line 1, in <module>
    from .inference import (async_inference_detector, inference_detector,
  File "/opt/conda/lib/python3.8/site-packages/mmdet/apis/inference.py", line 6, in <module>
    from mmcv.ops import RoIPool
  File "/opt/conda/lib/python3.8/site-packages/mmcv/ops/__init__.py", line 2, in <module>
    from .assign_score_withk import assign_score_withk
  File "/opt/conda/lib/python3.8/site-packages/mmcv/ops/assign_score_withk.py", line 5, in <module>
    ext_module = ext_loader.load_ext(
  File "/opt/conda/lib/python3.8/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
    ext = importlib.import_module('mmcv.' + name)
  File "/opt/conda/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: /opt/conda/lib/python3.8/site-packages/mmcv/_ext.cpython-38-x86_64-linux-gnu.so: undefined symbol: _Z27points_in_boxes_cpu_forwardN2at6TensorES0_S0_

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/demo/multimodial_demo.py", line 20, in <module>
    from demo.demo_skeleton import frame_extraction
  File "/workspace/phar/mmaction2/demo/demo_skeleton.py", line 18, in <module>
    raise ImportError('Failed to import `inference_detector` and '
ImportError: Failed to import `inference_detector` and `init_detector` form `mmdet.apis`. These apis are required in this demo!

[rlleshi]

Can you try to upgrade numpy to 1.23.0?

[davidhorrock0949]

I'll give it a shot this weekend and report back, thanks!

[AshoPash]

I didn't work for me . I've also get the hotfixes branch and here is the latest errors when I try to run multimodial_demo.py :

No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda
Traceback (most recent call last):
  File "/workspace/phar/mmaction2/demo/demo_skeleton.py", line 16, in <module>

    from mmcv.ops import RoIPool
  File "/opt/conda/lib/python3.8/site-packages/mmcv/ops/__init__.py", line 2, in <module>
    from .assign_score_withk import assign_score_withk
  File "/opt/conda/lib/python3.8/site-packages/mmcv/ops/assign_score_withk.py", line 5, in <module>
    ext_module = ext_loader.load_ext(
  File "/opt/conda/lib/python3.8/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext
    ext = importlib.import_module('mmcv.' + name)
  File "/opt/conda/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libcudart.so.10.2: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/demo/multimodial_demo.py", line 20, in <module>
    from demo.demo_skeleton import frame_extraction
  File "/workspace/phar/mmaction2/demo/demo_skeleton.py", line 18, in <module>
    raise ImportError('Failed to import `inference_detector` and '
ImportError: Failed to import `inference_detector` and `init_detector` form `mmdet.apis`. These apis are required in this demo!`

[larkinxp]

Had the exact same issue as @AshoPash and could not figure out how to resolve it.

[AndreasFrischCP]

@rlleshi when upgrading numpy to 1.23.0 inside the docker container, i could run the demo:

root@7edfcf328b82:/workspace/phar# pip install --force-reinstall numpy==1.23.0
Collecting numpy==1.23.0
  Downloading numpy-1.23.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
     |████████████████████████████████| 17.1 MB 1.8 MB/s 
Installing collected packages: numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 1.22.4
    Uninstalling numpy-1.22.4:
      Successfully uninstalled numpy-1.22.4
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

scikit-image 0.21.0 requires pillow>=9.0.1, but you'll have pillow 8.1.1 which is incompatible.
imageio 2.34.0 requires pillow>=8.3.2, but you'll have pillow 8.1.1 which is incompatible.
albumentations 1.4.0 requires numpy>=1.24.4, but you'll have numpy 1.23.0 which is incompatible.
Successfully installed numpy-1.23.0

root@7edfcf328b82:/workspace/phar# python src/demo/multimodial_demo.py /video.mp4 demo.json --timestamps
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
Resizing video for faster inference...
Moviepy - Building video temp/video.mp4.
MoviePy - Writing audio in videoTEMP_MPY_wvf_snd.mp3
MoviePy - Done.                                                                                                                                                                                 
Moviepy - Writing video temp/video.mp4

Moviepy - Done !                                                                                                                                                                                
Moviepy - video ready temp/video.mp4
load checkpoint from local path: checkpoints/har/timeSformer.pth
Traceback (most recent call last):
  File "src/demo/multimodial_demo.py", line 601, in <module>
    main()
  File "src/demo/multimodial_demo.py", line 540, in main
    RGB_MODEL = init_recognizer(args.rgb_config,
  File "/workspace/phar/mmaction2/mmaction/apis/inference.py", line 51, in init_recognizer
    model.to(device)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 673, in to
    return self._apply(convert)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 387, in _apply
    module._apply(fn)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 387, in _apply
    module._apply(fn)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 387, in _apply
    module._apply(fn)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 409, in _apply
    param_applied = fn(param)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 671, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
  File "/opt/conda/lib/python3.8/site-packages/torch/cuda/__init__.py", line 170, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

however, it didn't actually work because of CUDA problems with the container.

i couldn't get the demo to run outside a container because i couldn't find a working combination of corresponding versions between cuda, torch and mmcv which would work under ubuntu 22.04

[orangekittysoftware]

I have it working using numpy 1.23.0 as the only change to the software configuration; there was also a bug fix, #17. Beyond that, I have it running on the the Deep Learning AMI from AWS - the pyTorch Ubuntu 20.04 edition on g5.4xlarge instances, which is overpowered - g5.xlarge would do. The dlami is a "known working" combo of tools, it includes the nvidia-container-toolkit.

It works, but is woefully inaccurate on my data set. I'm looking for ways to visualize the output better.

[MartinDawson]

@orangekittysoftware Any progress on this? I'm also going to try the same thing as I don't have a NVIDIA GPU.

Why is it so inaccurate on your dataset?

[orangekittysoftware]

@orangekittysoftware Any progress on this? I'm also going to try the same thing as I don't have a NVIDIA GPU. Well, running that special g5.x4large was expensive and delicate so I do it occasionally. Must be shut off after use. It was fast though!

I found I could run about 4-6 videos at a time in a g5.4xlarge at 480p. I did not try much more or try to stress test it.

Anyway, I may have the chance to try running it locally on NVIDIA hardware.

Why is it so inaccurate on your dataset?

Ahem, "dataset bias due to preference"? I'll speak a bit more openly to it because I think it speaks to a modeling issue: The existing models for PHAR are hardcore - based on two people. But a lot of porn is solo / softcore - which also has some different problems for HAR. I'd be interested in a side project to create a PHAR-like model and actionset set for softcore.

[MartinDawson]

Makes sense, Thanks. I'm going to get a NVidia GPU and will see what the results are once i have it hopefully by end of week, this repo is still the most promising multi-label porn solution I have seen so far.

Tried to use AWS but they don't allow you to use g-family EC2 instances for new AWS accounts anymore due to crypto miners abusing it and I lost my old account :(

I'm wanting to extend this repo and train it on specific fetishes as I think this is something that is not being addressed by the existing ML solutions by mindgeek and other providers that currently use ML.

[MartinDawson]

Got it working, with thanks from @orangekittysoftware suggestion and fix PR, seems to work well on a simple oral video.

By the way, I am using vast.ai to rent cheap RTX GPU's instead of AWS. It's much cheaper. Would recommend it.