[CODEGEN-1918] Support concurrent requests for Custom Models

[yura-ant]

This repository is public. Do not put here any private DataRobot or customer's data: code, datasets, model artifacts, .etc.

Summary

Allow enable flask process via command line option --max_workers or define CUSTOM_MODEL_WORKERS numeric runtime parameter Option 3 from https://docs.google.com/document/d/1yq-T7-pmjW49RsvbslNIl3BF_GRbYLti2McVmZgd_To/edit#heading=h.8vln2nt42rgh

TODO I would like to add more test. But in general I think it is ready for review

Rationale

[yura-ant]

jarvis please test failed

[yura-ant]

jenkins run test_functional_by_framework-check_by_framework

[devexp-slackbot[bot]]

The Needs Review labels were added based on the following file changes.

Team @datarobot/core-modeling (#core-modeling) was assigned because of changes in files:

custom_model_runner/datarobot_drum/drum/args_parser.py
custom_model_runner/datarobot_drum/drum/drum.py
custom_model_runner/datarobot_drum/drum/main.py
custom_model_runner/datarobot_drum/resource/components/Python/prediction_server/prediction_server.py
custom_model_runner/datarobot_drum/resource/drum_server_utils.py
custom_model_runner/datarobot_drum/resource/pipelines/prediction_server_pipeline.json.j2
custom_model_runner/datarobot_drum/runtime_parameters/runtime_parameters.py
tests/functional/test_inference_per_framework.py

Team @datarobot/custom-models (#custom-models) was assigned because of changes in files:

custom_model_runner/datarobot_drum/drum/args_parser.py
custom_model_runner/datarobot_drum/drum/drum.py
custom_model_runner/datarobot_drum/drum/main.py
custom_model_runner/datarobot_drum/resource/components/Python/prediction_server/prediction_server.py
custom_model_runner/datarobot_drum/resource/drum_server_utils.py
custom_model_runner/datarobot_drum/resource/pipelines/prediction_server_pipeline.json.j2
custom_model_runner/datarobot_drum/runtime_parameters/runtime_parameters.py
tests/functional/test_inference_per_framework.py
tests/unit/datarobot_drum/drum/test_args_parser.py
tests/unit/datarobot_drum/drum/test_drum.py
tests/unit/datarobot_drum/drum/test_prediction_server.py
tests/unit/datarobot_drum/runtime_parameters/test_runtime_parameters.py

If you think that there are some issues with ownership, please discuss with C&A domain at #core-backend-domain slack channel and create PR to update DRCODEOWNERS\CODEOWNERS file.

[devexp-slackbot[bot]]

Label Needs Review: Core Modeling was removed because @scottp-ml is part of Core Modeling domain.

[yura-ant]

Hi @yakov-g, Have you had a chance to look at this?