ModuleNotFoundError: No module named 'scenedetect' in Container

[nghtm]

When running scene_detect.py within the default docker container, I encounter an error ModuleNotFoundError: No module named 'scenedetect'.

To resolve, I run pip install scenedetect within the container. The scenedetect script then fails with:

Traceback (most recent call last):
  File "/opt/conda/envs/pytorch/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/envs/pytorch/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/tools/scene_cut/scene_detect.py", line 69, in <module>
    main()
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/tools/scene_cut/scene_detect.py", line 56, in main
    ret = meta.parallel_apply(process_single_row, axis=1)
  File "/opt/conda/envs/pytorch/lib/python3.10/site-packages/pandarallel/core.py", line 294, in closure
    pool = CONTEXT.Pool(nb_workers)
  File "/opt/conda/envs/pytorch/lib/python3.10/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/opt/conda/envs/pytorch/lib/python3.10/multiprocessing/pool.py", line 205, in __init__
    raise ValueError("Number of processes must be at least 1")
ValueError: Number of processes must be at least 1

I have isolated the issue, I am able to get pandaparallelize to recognize number of workers, and the attached test script executes successfully in the container. It seems to be due to some conflict with interaction between Pandarallel and the other components of your script (like scenedetect or how DataFrame is being manipulated).

[nghtm]

sanity-check-py.txt

[nghtm]

was able to resolve by following steps in https://github.com/hpcaitech/Open-Sora/blob/main/docs/installation.md