github-actions[bot]
commented
1 year ago 👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.
Access additional YOLOv5 🚀 resources:
- Wiki – https://github.com/ultralytics/yolov5/wiki
- Tutorials – https://docs.ultralytics.com/yolov5
- Docs – https://docs.ultralytics.com
Access additional Ultralytics ⚡ resources:
- Ultralytics HUB – https://ultralytics.com/hub
- Vision API – https://ultralytics.com/yolov5
- About Us – https://ultralytics.com/about
- Join Our Team – https://ultralytics.com/work
- Contact Us – https://ultralytics.com/contact
Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!
Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!
glenn-jocher
Search before asking
Question
I am running Yolov5 in Azure Machine Learning instead of Google Colab. Microsoft has a full featured integrated capability of creating docker containers, and run it on GPU's in the cloud straight from Visual Studio Code. 0.23 USD/ hr for a T80 Tesla on low priority infrastructure. It even has full github ML Ops integrated. See image below where I show it is working...but with some hacks in your code. It is much easier to use than Google CoLab, but getting it to work with this repository is a bit hard. I particular as I do not know how to avoid mixing up our code with this repository, but that is maybe because Python is not that OO as C# to have it perfectly separated.
I modified this in the requirements.txt: opencv-python-headless>=4.1.1
opencv-python>=4.1.1
My script is a py file in the Yolo folder. Basically this is all needed to fully create a container and sent it to the Azure ML cloud: `from pathlib import Path
from azureml.core import ( Environment, Experiment, Run, ScriptRunConfig, Workspace, Dataset, Datastore, ) from azureml.data import OutputFileDatasetConfig
from azureml.core.compute import AmlCompute, ComputeTarget from azureml.core.authentication import AzureCliAuthentication
def find_or_create_compute_target( workspace, name, vm_size="Standard_NC6", #"Standard_D8_v3", min_nodes=0, max_nodes=1, idle_seconds_before_scaledown=900, vm_priority="lowpriority", ):
workspace_config = 'config.json' requirements_file = 'requirements.txt' compute_target_name = 'LowPrioStdNC6' experiment_name = 'train-yolo-7' scriptpath = 'segment/train.py' blobstorage = 'workspaceblobstore' blobcontainerpath = 'UI/2023-01-05###_UTC'
cli_auth = AzureCliAuthentication()
Authenticate with your AzureML Resource via its config.json file
ws = Workspace.from_config(path=workspace_config, auth=cli_auth)
ws = Workspace.from_config(path="v4a/config.json", auth=cli_auth)
The experiment in this workspace under which our runs will be grouped
If an experiment with the given name doesn't exist, it will be created
exp = Experiment(ws, experiment_name)
The compute cluster you want to run on and its settings.
If it doesn't exist, it'll be created.
compute_target = find_or_create_compute_target(ws, compute_target_name)
The Environment lets us define any dependencies needed to make our script run
env = Environment.from_pip_requirements("my-pip-env", requirements_file)
datastore = Datastore.get(ws, blobstorage) dataset = Dataset.File.from_files(path=(datastore, blobcontainerpath))
output = OutputFileDatasetConfig(name='outputs', destination=(datastore, 'outputdataset'))
script_args = ['--device', '0', '--img', '640', '--cfg', 'v4a/data/yolov5s-seg.yaml', '--hyp', 'data/hyps/hyp.scratch-high.yaml', '--batch', '16', '--epochs', '100', '--data', 'v4a/data/###.yaml', '--weights', 'v4a/data/yolov5s-seg.pt', '--workers', '24', '--name', '#####', '--project', output.as_mount(), '--inputs', dataset.as_named_input('inputs').as_mount()]
'--outputs', output.as_mount()]
A run configuration is how you define what youd like to run
We give it the directory where our code is, the script we want to run, the environment, and the compute info
run_config = ScriptRunConfig( source_directory=".",#Path(script_path).parent, script="./segment/train.py", #Path(script_path).name, arguments=script_args, compute_target=compute_target, environment=env, )
Submit our configured run under our experiment
run = exp.submit(run_config)
Wait for the run to finish
run.wait_for_completion(show_output=True)
Delete our compute target so it doesn't cost us money
compute_target.delete()`
The input and outputs are explained in this Jupyter notebook: https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/work-with-data/datasets-tutorial/scriptrun-with-data-input-output/how-to-use-scriptrun.ipynb
As our dataset is too large we need to mount the dataset from an azure blob storage. I can however not see how to change the path. I see in the yaml file a path can be set, but that is not working.
Is it possible to support this natively ? I am a C# programmer and do not get the idea of OO in Python. So I did some modifications which are not so nice:
Another not so nice:
It is currently processing, question is if I can see the outputs of the training somewhere:
The output is also saved to a azure blob:
Additional
Would be nice to support Microsoft Azure Machine Learning natively including azure blob storage for large files. It will save the environment by not having to pump data around.