search

CloudVE / cloudbridge

A consistent interface to multiple IaaS clouds; in Python.
https://cloudbridge.cloudve.org
MIT License
113 stars 51 forks source link

Fix issues with upload blob, blob url and snapshots #282

Closed FabioRosado closed 2 years ago

FabioRosado commented 2 years ago

I've seeing some weird things locally and going through each bit is taking a bit too long :disappointed:

This new PR fixes issues with the blob - I also add to add the Storage Blob Contributor role in order to get the objects inside a storage bucket.

Getting the signed URL was also broken so I fixed it in this PR. Creating a snapshot from a volume wasn't working anymore because create_or_update expects you to pass a snapshot class. This PR adds the Disk object to the API, but there are some issues, for example, I didn't yet figure out how to get the new tag into the snapshot so our fetching will be broken :disappointed:

A note on the AzureBucketObject, when we are listing the objects, Azure returns a BlobProperties, but when we are getting it returns the blobclient. The easiest way to handle this was to add self._blob_properties to the class and use that instead

nuwang commented 2 years ago

Thanks for continuing to hack away at this @FabioRosado. I got my institutional Azure credentials sorted out a few days ago, so I'm hoping to join the fray in the next couple of days. Will keep you posted.

FabioRosado commented 2 years ago

No worries happy to :smile:

Today I spent some time looking into the Azure API but couldn't figure out how to replace the tag in a single command, so I am calling update_snapshot_tags when calling create_snapshot and confirmed tha the new snapshot will have the right tag. Still feels a bit hacky though :disappointed:

One thing that I think might be worth mentioning. I've ran a bunch of tests manually (or my understanding of the crud ones) and manually they all work fine. But when running them with tox they keep erroring with the error 'resource not found'. I suspect the reason for this, is because Azure actually takes a long time to create things like network resources, firewalls, etc.

Do you think the issue could be timing and perhaps we need to wait for the resource to be created? :thinking: Haven't dug much into the tests yet tbh.

On an unrelated note, would you be open to adding the hacktoberfest topic to the project? I'm okay if you prefer not, I still have nightmares from last year spam PRs haha

nuwang commented 2 years ago

@FabioRosado I think the two step process sounds fine if there's no way to do it in one go any more.

Regarding the resource group wait, that does sound plausible. I thought there was a wait, but perhaps the new API needs a manual wait.

Have added the hacktoberfest topic to the project, let me know whether that looks right.