search

skypilot-org / skypilot

SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.
https://skypilot.readthedocs.io
Apache License 2.0
6.82k stars 514 forks source link

[Azure] Update azure dependencies in setup.py #4202

Closed romilbhardwaj closed 3 weeks ago

romilbhardwaj commented 4 weeks ago

After #4319, we need to pin the min azure sdk version in setup.py. On azure-core==1.30.0 (pip freeze), it fails with:

E 10-28 19:02:54 instance.py:537] Failed to create instances: [AttributeError] 'SubResource' object has no attribute 'validate'
D 10-28 19:02:54 provisioner.py:150] Failed to provision 'azure' on Azure (all zones).
D 10-28 19:02:54 provisioner.py:152] bulk_provision for 'azure' failed. Stacktrace:
D 10-28 19:02:54 provisioner.py:152] Traceback (most recent call last):
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/provision/provisioner.py", line 141, in bulk_provision
D 10-28 19:02:54 provisioner.py:152]     return _bulk_provision(cloud, region, cluster_name,
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/provision/provisioner.py", line 63, in _bulk_provision
D 10-28 19:02:54 provisioner.py:152]     provision_record = provision.run_instances(provider_name,
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/provision/__init__.py", line 50, in _wrapper
D 10-28 19:02:54 provisioner.py:152]     return impl(*args, **kwargs)
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/provision/azure/instance.py", line 525, in run_instances
D 10-28 19:02:54 provisioner.py:152]     created_instances = _create_instances(
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/provision/azure/instance.py", line 355, in _create_instances
D 10-28 19:02:54 provisioner.py:152]     subprocess_utils.run_in_parallel(create_single_instance, range(count))
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/utils/subprocess_utils.py", line 65, in run_in_parallel
D 10-28 19:02:54 provisioner.py:152]     return list(p.imap(func, args))
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/multiprocessing/pool.py", line 870, in next
D 10-28 19:02:54 provisioner.py:152]     raise value
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/multiprocessing/pool.py", line 125, in worker
D 10-28 19:02:54 provisioner.py:152]     result = (True, func(*args, **kwds))
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/provision/azure/instance.py", line 350, in create_single_instance
D 10-28 19:02:54 provisioner.py:152]     network_interface = _create_network_interface(network_client, vm_name,
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/Romil/Berkeley/Research/sky-experiments/sky/provision/azure/instance.py", line 226, in _create_network_interface
D 10-28 19:02:54 provisioner.py:152]     ni_poller = network_client.network_interfaces.begin_create_or_update(
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
D 10-28 19:02:54 provisioner.py:152]     return func(*args, **kwargs)
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/azure/mgmt/network/v2022_01_01/operations/_operations.py", line 30919, in begin_create_or_update
D 10-28 19:02:54 provisioner.py:152]     raw_result = self._create_or_update_initial(  # type: ignore
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/azure/mgmt/network/v2022_01_01/operations/_operations.py", line 30833, in _create_or_update_initial
D 10-28 19:02:54 provisioner.py:152]     _json = self._serialize.body(parameters, 'NetworkInterface')
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/msrest/serialization.py", line 628, in body
D 10-28 19:02:54 provisioner.py:152]     errors = _recursive_validate(data_type, data_type, data)
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/msrest/serialization.py", line 160, in _recursive_validate
D 10-28 19:02:54 provisioner.py:152]     return data.validate()
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/msrest/serialization.py", line 254, in validate
D 10-28 19:02:54 provisioner.py:152]     validation_result += _recursive_validate(attr_name, attr_type, value)
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/msrest/serialization.py", line 153, in _recursive_validate
D 10-28 19:02:54 provisioner.py:152]     result += _recursive_validate(attr_name+" content", attr_type[1:-1], content)
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/msrest/serialization.py", line 160, in _recursive_validate
D 10-28 19:02:54 provisioner.py:152]     return data.validate()
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/msrest/serialization.py", line 254, in validate
D 10-28 19:02:54 provisioner.py:152]     validation_result += _recursive_validate(attr_name, attr_type, value)
D 10-28 19:02:54 provisioner.py:152]   File "/Users/romilb/tools/anaconda3/lib/python3.9/site-packages/msrest/serialization.py", line 160, in _recursive_validate
D 10-28 19:02:54 provisioner.py:152]     return data.validate()
D 10-28 19:02:54 provisioner.py:152] AttributeError: 'SubResource' object has no attribute 'validate'
romilbhardwaj commented 3 weeks ago

Closed with #4204.