[RAPTOR-11322] Add lazy loading processor to download the associated lazy loading remote files to the local volume

[zohar-mizrahi]

Summary

In the context of the lazy loading custom model support, it is required to process the related environment variable, if exists, and download the required files to the root folder of the executed custom model.

Changes

• Add a new method to the BaseLanguagePredictor to handle lazy loading files and call it from the PythonPredictor within the configuration phase.
• Add a new handler LazyLoadingHandler to deal with processing the lazy loading structured data JSON from an environment variable and use datarobot_storage to download the lazy loading files in parallel.
• Small improvement to the credential_type attribute definition in S3Credentials.
• Minor change to .gitconfig.
• Unit, integration and functional tests - note that the functional test requires special environment variables in order not to be skipped. It assumes an accessible MinIO service that is set with specific files. (I run it locally in my environment).

[engprod-2[bot]]

The Needs Review labels were added based on the following file changes.

Team @datarobot/core-modeling (#core-modeling) was assigned because of changes in files:

custom_model_runner/datarobot_drum/drum/language_predictors/base_language_predictor.py
custom_model_runner/datarobot_drum/drum/language_predictors/python_predictor/python_predictor.py
custom_model_runner/datarobot_drum/drum/lazy_loading/constants.py
custom_model_runner/datarobot_drum/drum/lazy_loading/exceptions.py
custom_model_runner/datarobot_drum/drum/lazy_loading/lazy_loading_handler.py
custom_model_runner/datarobot_drum/drum/lazy_loading/schema.py
custom_model_runner/requirements.txt
tests/functional/test_lazy_loading.py

Team @datarobot/custom-models (#custom-models) was assigned because of changes in files:

.gitignore
custom_model_runner/datarobot_drum/drum/language_predictors/base_language_predictor.py
custom_model_runner/datarobot_drum/drum/language_predictors/python_predictor/python_predictor.py
custom_model_runner/datarobot_drum/drum/lazy_loading/constants.py
custom_model_runner/datarobot_drum/drum/lazy_loading/exceptions.py
custom_model_runner/datarobot_drum/drum/lazy_loading/lazy_loading_handler.py
custom_model_runner/datarobot_drum/drum/lazy_loading/schema.py
custom_model_runner/requirements.txt
tests/functional/test_lazy_loading.py
tests/integration/datarobot_drum/drum/language_predictors/test_language_predictors.py
tests/integration/datarobot_drum/drum/lazy_loading/__init__.py
tests/integration/datarobot_drum/drum/lazy_loading/test_lazy_loading.py
tests/unit/datarobot_drum/drum/language_predictors/python_predictor/test_python_predictor.py

If you think that there are some issues with ownership, please discuss with C&A domain at #core-backend-domain slack channel and create PR to update DRCODEOWNERS\CODEOWNERS file.