Failure to correctly parse SAS URL for Managed Disk

kf6kjg commented 1 year ago

Problem Description

When given a SAS URL from the command az disk grant-access a ValueError is raised: ValueError: Invalid derived mode from --storage-url: z31

Steps to Reproduce

Have a local VHD file ready to upload to an Azure Managed Disk, and said disk created and ready for upload.
Execute something akin to the following:

< path/to/abcd.vhd \
blobxfer upload \
  --local-path - \
  --overwrite \
  --storage-url "$(
    az disk grant-access \
      --duration-in-seconds ${10 * 60 * 60} \
      --access-level Write \
      --ids "${REDACTED}" \
      --output tsv \
      --query "accessSas" \
      ;
  )"

And at this point I'm breaking from your template because I've already found the code that's problematic.

Resulting error

  File "/usr/local/bin/blobxfer", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/blobxfer_cli/cli.py", line 1128, in upload
    settings.add_cli_options(ctx.cli_options, settings.TransferAction.Upload)
  File "/usr/local/lib/python3.9/dist-packages/blobxfer_cli/settings.py", line 77, in add_cli_options
    raise ValueError(
ValueError: Invalid derived mode from --storage-url: z31

Expected Result

The file to upload, barring other issues outside the scope of this tool.

Problematic code

See https://github.com/Azure/blobxfer/blob/0ac1212326a43dfd6cb2b8525ff95f1c4ae540af/blobxfer/util.py#LL343C5-L343C12

The format of the SAS URL coming from the one instance of az disk grant-access that I've been working with is:

https://md-impexp-r3dacted.z31.blob.storage.azure.net/redact3d/abcd?sv=2018-03-28&sr=b&si=REDACTED-UUID&sig=REDACTED%3D

As you can see, this results in the code below incorrectly assuming the mode is in the second-from-the-left subdomain. A better assumption would be that the mode is in the 4th from the right.

    tmp = url.split('/')
    host = tmp[2].split('.')
    sa = host[0]
    mode = host[1].lower()

My barely-adequate knowledge of snakelang and, more importantly, limited time and knowledge of your code base and practices leaves me with raising this issue ticket instead of a PR. :)

kf6kjg commented 1 year ago

Working on a PR anyway. hotpatching my local install isn't viable for what I'm working on.

kf6kjg commented 1 year ago

Note: I found a different way to solve my core issue - I shrunk the managed disks by copying to new smaller disks since the source disks were only ~5% full. This allowed me to just use azcopy since I didn't need to pipe the file through gzip. Thus I'm leaving this issue and PR #145 here, but I'm stepping away from maintaining them. :/

Azure / blobxfer