microsoft / Oscar

Oscar and VinVL
MIT License
1.04k stars 251 forks source link

Public access for pretrained models are blocked #205

Closed HireTheHero closed 1 year ago

HireTheHero commented 1 year ago

Hi, Following DOWNLOAD.md, pretrained model download command wget https://biglmdiag.blob.core.windows.net/oscar/pretrained_models/$MODEL_NAME.zip throws the following error.

--2023-06-11 04:32:29--  https://biglmdiag.blob.core.windows.net/oscar/pretrained_models/large-vg-labels.zip
Resolving biglmdiag.blob.core.windows.net (biglmdiag.blob.core.windows.net)... 52.239.207.225
Connecting to biglmdiag.blob.core.windows.net (biglmdiag.blob.core.windows.net)|52.239.207.225|:443... connected.
HTTP request sent, awaiting response... 409 Public access is not permitted on this storage account.
2023-06-11 04:32:30 ERROR 409: Public access is not permitted on this storage account..

The message above is for large-vg-labels but it's same for the all models. Could you check your settings?

### Tasks
- [ ] Access to the pretrained models should be permitted
Feve1986 commented 1 year ago

@HireTheHero Hi there! I came across the same issue that you mentioned here. Were you able to find an alternative link or a solution to access the file? Your help would be much appreciated. Thanks in advance!

ChunyuanLI commented 1 year ago

Note: The data is on Azure Storage Blob, a SAS with Read permission is provided. Please append the following SAS at the end of each link to download:

?sp=r&st=2023-08-28T01:12:41Z&se=3023-08-28T09:12:41Z&sv=2022-11-02&sr=c&sig=6R1YmWluiXmPLsdVn1rDUpeBp2SYBMxDjc6KoKNlY8Q%3D
xuehaoliu109 commented 1 year ago

Note: The data is on Azure Storage Blob, a SAS with Read permission is provided. Please append the following SAS at the end of each link to download:

?sp=r&st=2023-08-28T01:12:41Z&se=3023-08-28T09:12:41Z&sv=2022-11-02&sr=c&sig=6R1YmWluiXmPLsdVn1rDUpeBp2SYBMxDjc6KoKNlY8Q%3D

Could you please update the VinVL_DOWNLOAD.md file, too? Thank you very much.

Rosiekk commented 1 year ago

Note: The data is on Azure Storage Blob, a SAS with Read permission is provided. Please append the following SAS at the end of each link to download:

?sp=r&st=2023-08-28T01:12:41Z&se=3023-08-28T09:12:41Z&sv=2022-11-02&sr=c&sig=6R1YmWluiXmPLsdVn1rDUpeBp2SYBMxDjc6KoKNlY8Q%3D

Thanks for the SAS token! However, when I appended the SAS token to download the pre-extracted features in the VinVL_DOWNLOAD.md file, I met the following issues: ./azcopy cp "https://biglmdiag.blob.core.windows.net/vinvl/image_features/oi_X152C4_frcnnbig2_exp168model_0060000model.roi_heads.nm_filter_2_model.roi_heads.score_thresh_0.2/model_0060000?sp=r&st=2023-08-28T01:12:41Z&se=3023-08-28T09:12:41Z&sv=2022-11-02&sr=c&sig=6R1YmWluiXmPLsdVn1rDUpeBp2SYBMxDjc6KoKNlY8Q%3D" [local_path] --recursive

INFO: Scanning...
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support

failed to perform copy command due to error: cannot start job due to error: cannot list files due to reason -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /home/vsts/go/pkg/mod/github.com/!azure/azure-storage-blob-go@v0.15.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=AuthenticationFailed) =====
Description=403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature., Details: (none)
   HEAD https://biglmdiag.blob.core.windows.net/vinvl/image_features/oi_X152C4_frcnnbig2_exp168model_0060000model.roi_heads.nm_filter_2_model.roi_heads.score_thresh_0.2/model_0060000?se=3023-08-28t09%3A12%3A41z&sig=-REDACTED-&sp=r&sr=c&st=2023-08-28t01%3A12%3A41z&sv=2022-11-02&timeout=901
   User-Agent: [AzCopy/10.20.1 Azure-Storage/0.15 (go1.19.12; linux)]
   X-Ms-Client-Request-Id: [aa911033-29be-4949-4899-d28b11af8e85]
   X-Ms-Version: [2020-10-02]
   --------------------------------------------------------------------------------
   RESPONSE Status: 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
   Date: [Sat, 02 Sep 2023 08:18:05 GMT]
   Server: [Microsoft-HTTPAPI/2.0]
   X-Ms-Error-Code: [AuthenticationFailed]
   X-Ms-Request-Id: [6f4fb951-c01e-00f1-0475-dd6d29000000]

Any ideas? Thank you very much!

HireTheHero commented 1 year ago

@ChunyuanLI Works for me, thanks. @Rosiekk I'm closing this issue since manual file transfer should be fine in my case. Feel free to reopen it if you wish to continue the discussion about azcopy.