this is the base functionality required to run any linux cmd-line tool within an AML pipeline step. For now I sticked to the structure MLOpsPython provides e.g. putting all script paths as variables.
[X] Adding custom AML Pipeline compatible plain Dockerfile
[X] Adding AML pipeline running on custom docker and executing an OS cmd wrapper script
[X] Adding OS cmd wrapper script preprocess_os_cmd.py leveraging subprocess to copy files from input folder to output folder
[X] Adding new environment variables to env.example
[X] Modify util.manage_environments.py to allow Dockerfile usage
[X] Modify util.manage_environments.py to correctly update environments
[ ] Adding unit tests to test preprocess and pipeline script --> please see #21 for direction discussion, therefore won't add unit test to this PR
[x] Adding AzDO pipelines to verify Dockerfile
[x] Adding AzDO pipelines to publish new AML pipeline
[x] Documentation on how to modify to install a tool and use it instead of cp
Hi,
this is the base functionality required to run any linux cmd-line tool within an AML pipeline step. For now I sticked to the structure MLOpsPython provides e.g. putting all script paths as variables.
preprocess_os_cmd.pyleveragingsubprocessto copy files from input folder to output folderenv.exampleutil.manage_environments.pyto allow Dockerfile usageutil.manage_environments.pyto correctly update environmentscp