Warning with sklearn parallel jobs

[slhck]

There is a warning running the default command:

UserWarning: Loky-backed parallel loops cannot be called in a multiprocessing, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)

I am not sure if this is really a problem, as the code seems to run well nonetheless!

Full log:

➜ poetry run p1204_3 test_videos/test_video_h264.mkv
INFO:root:handle the following videos (# 1): 
  test_videos/test_video_h264.mkv
INFO:root:check or install video parser
INFO:root:video parser is checked out
INFO:root:video parser is build
INFO:root:use feature cache file ./tmp/test_video_h264_feat.pkl
INFO:root:features are already cached, extraction skipped
INFO:root:features extracted
/home/werner/.cache/pypoetry/virtualenvs/p1204-3-Ryi-Q4po-py3.8/lib/python3.8/site-packages/sklearn/ensemble/_base.py:175: UserWarning: Loky-backed parallel loops cannot be called in a multiprocessing, setting n_jobs=1
  n_jobs = min(effective_n_jobs(n_jobs), n_estimators)
[
    {
        "date": "2020-04-16 12:14:13.100134",
        "debug": {
            "baseline": 4.16292374098929,
            "coding_deg": 32.43006645212775,
            "rf_pred": 4.010545689643026,
            "temporal_deg": 0.0,
            "upscaling_deg": 0.0
        },
        "per_second": [
            4.132547886568417,
            3.931621503558015,
            4.328687969986304,
            4.3002566462931,
            4.1819097998511054,
            3.9471693619756834,
            3.935126424508243,
            3.9407565516571026,
            3.955412313745017,
            3.991729591232866
        ],
        "per_sequence": 4.086734715316158,
        "video_basename": "test_video_h264.mkv",
        "video_full_path": "test_videos/test_video_h264.mkv"
    }
]
INFO:root:store all results to reports

[stg7]

i think it is because the model uses "all cores", whereas it also uses multiprocessing, however this is not creating any issues during running the models, so i will check this later

[slhck]

Feel free to close this, if it does not matter much! We should maybe somehow hide the warning though, because it could be confusing. And then it is not possible to a clean JSON output when using the --quiet flag.

[stg7]

fixed in 782657aef7898a892ca40521199c0c3ff278903e

[stg7]

just for documentation, the warning occurs because of the n_jobs=-1 setting and multiprocessing, however this is only required for training, thus can be changed to 1 for prediction

[slhck]

Cool, thanks!