kpfaulkner / azurecopy

copy blobs between azure, s3 and local storage
Apache License 2.0
36 stars 13 forks source link

error while copying data, list is working fine. #37

Open Sonam-sunny opened 6 years ago

Sonam-sunny commented 6 years ago

I am getting error when i am trying copy data from s3 to azure blob. but i am able to list data from both amazon s3 and azure blob. error types are 404 and 403.

kpfaulkner commented 6 years ago

Hi

Do you get any stacktrace at all?

Also, can you show me the exact command you used?

Thanks

Ken

Sonam-sunny commented 6 years ago

command- azurecopy.exe -i "https:// s3.amazonaws.com/name/receipts" -o "C:\Azureifcopies" -azu rekey %AzureAccountKey% -s3k %AWSAccessKeyID% -s3sk %AWSSecretAccessKeyID% -blob copy -destblobtype block

Error log- GetHandler start GetHandler retrieved azurecopy.S3Handler GetHandler start GetHandler retrieved azurecopy.FileSystemHandler Unable to start copying receipts Exception System.ArgumentException: The argument must not be empty string. Parameter name: accountName at Microsoft.WindowsAzure.Storage.Core.Util.CommonUtility.AssertNotNullOrEmpt y(String paramName, String value) at Microsoft.WindowsAzure.Storage.Auth.StorageCredentials..ctor(String accoun tName, String keyValue, String keyName) at azurecopy.Utils.AzureHelper.GetCloudStorageAccount(String url, String acco untKey, String accountName) at azurecopy.Utils.AzureHelper.GetCloudBlobClient(String url, Boolean isSrc, String accountKey) at azurecopy.AzureBlobCopyHandler.StartCopy(BasicBlobContainer origBlob, Stri ng DestinationUrl, DestinationBlobType destBlobType, Boolean skipIfExists) at azurecopy.AzureBlobCopyHandler.StartCopyList(IEnumerable1 origBlobList, S tring destinationUrl, DestinationBlobType destBlobType, Boolean debugMode, Boole an skipIfExists) New Batch Unknown error generated. Please report to Github page https://github.com/kpfaulk ner/azurecopy/issues . Can view underlying stacktrace by adding -db flag. Syste m.ArgumentException: The argument must not be empty string. Parameter name: accountName at Microsoft.WindowsAzure.Storage.Core.Util.CommonUtility.AssertNotNullOrEmpt y(String paramName, String value) at Microsoft.WindowsAzure.Storage.Auth.StorageCredentials..ctor(String accoun tName, String keyValue, String keyName) at azurecopy.Utils.AzureHelper.GetCloudStorageAccount(String url, String acco untKey, String accountName) at azurecopy.Utils.AzureHelper.GetCloudBlobClient(String url, Boolean isSrc, String accountKey) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl, Copy Status copyStatusFilter) at azurecopy.AzureBlobCopyHandler.MonitorBlobCopy(String destinationUrl) at azurecopy.AzureBlobCopyHandler.StartCopyList(IEnumerable1 origBlobList, S tring destinationUrl, DestinationBlobType destBlobType, Boolean debugMode, Boole an skipIfExists) at azurecopycommand.Program.DoNormalCopy(Boolean debugMode) at azurecopycommand.Program.Main(String[] args) at Microsoft.WindowsAzure.Storage.Core.Util.CommonUtility.AssertNotNullOrEmpt y(String paramName, String value) at Microsoft.WindowsAzure.Storage.Auth.StorageCredentials..ctor(String accoun tName, String keyValue, String keyName) at azurecopy.Utils.AzureHelper.GetCloudStorageAccount(String url, String acco untKey, String accountName) at azurecopy.Utils.AzureHelper.GetCloudBlobClient(String url, Boolean isSrc, String accountKey) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl, Copy Status copyStatusFilter) at azurecopy.AzureBlobCopyHandler.MonitorBlobCopy(String destinationUrl) at azurecopy.AzureBlobCopyHandler.StartCopyList(IEnumerable`1 origBlobList, S tring destinationUrl, DestinationBlobType destBlobType, Boolean debugMode, Boole an skipIfExists) at azurecopycommand.Program.DoNormalCopy(Boolean debugMode) at azurecopycommand.Program.Main(String[] args)

On Tue, May 22, 2018 at 7:57 PM, Ken Faulkner notifications@github.com wrote:

Hi

Do you get any stacktrace at all?

Also, can you show me the exact command you used?

Thanks

Ken

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/kpfaulkner/azurecopy/issues/37#issuecomment-391120353, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZZYOOUo6rGoMN81ljx73rEURyIfn-9ks5t1G2VgaJpZM4UI2mF .

Sonam-sunny commented 6 years ago

Hi Ken,

Please find the following log.\

GetHandler retrieved azurecopy.S3Handler GetHandler start GetHandler retrieved azurecopy.FileSystemHandler Unable to start copying receipts Exception System.ArgumentException: The argument must not be empty string. Parameter name: accountName at Microsoft.WindowsAzure.Storage.Core.Util.CommonUtility.AssertNotNullOrEmpt y(String paramName, String value) at Microsoft.WindowsAzure.Storage.Auth.StorageCredentials..ctor(String accoun tName, String keyValue, String keyName) at azurecopy.Utils.AzureHelper.GetCloudStorageAccount(String url, String acco untKey, String accountName) at azurecopy.Utils.AzureHelper.GetCloudBlobClient(String url, Boolean isSrc, String accountKey) at azurecopy.AzureBlobCopyHandler.StartCopy(BasicBlobContainer origBlob, Stri ng DestinationUrl, DestinationBlobType destBlobType, Boolean skipIfExists) at azurecopy.AzureBlobCopyHandler.StartCopyList(IEnumerable1 origBlobList, S tring destinationUrl, DestinationBlobType destBlobType, Boolean debugMode, Boole an skipIfExists) New Batch Unknown error generated. Please report to Github page https://github.com/kpfaulk ner/azurecopy/issues . Can view underlying stacktrace by adding -db flag. Syste m.ArgumentException: The argument must not be empty string. Parameter name: accountName at Microsoft.WindowsAzure.Storage.Core.Util.CommonUtility.AssertNotNullOrEmpt y(String paramName, String value) at Microsoft.WindowsAzure.Storage.Auth.StorageCredentials..ctor(String accoun tName, String keyValue, String keyName) at azurecopy.Utils.AzureHelper.GetCloudStorageAccount(String url, String acco untKey, String accountName) at azurecopy.Utils.AzureHelper.GetCloudBlobClient(String url, Boolean isSrc, String accountKey) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl, Copy Status copyStatusFilter) at azurecopy.AzureBlobCopyHandler.MonitorBlobCopy(String destinationUrl) at azurecopy.AzureBlobCopyHandler.StartCopyList(IEnumerable1 origBlobList, S tring destinationUrl, DestinationBlobType destBlobType, Boolean debugMode, Boole an skipIfExists) at azurecopycommand.Program.DoNormalCopy(Boolean debugMode) at azurecopycommand.Program.Main(String[] args) at Microsoft.WindowsAzure.Storage.Core.Util.CommonUtility.AssertNotNullOrEmpt y(String paramName, String value) at Microsoft.WindowsAzure.Storage.Auth.StorageCredentials..ctor(String accoun tName, String keyValue, String keyName) at azurecopy.Utils.AzureHelper.GetCloudStorageAccount(String url, String acco untKey, String accountName) at azurecopy.Utils.AzureHelper.GetCloudBlobClient(String url, Boolean isSrc, String accountKey) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl, Copy Status copyStatusFilter) at azurecopy.AzureBlobCopyHandler.MonitorBlobCopy(String destinationUrl) at azurecopy.AzureBlobCopyHandler.StartCopyList(IEnumerable`1 origBlobList, S tring destinationUrl, DestinationBlobType destBlobType, Boolean debugMode, Boole an skipIfExists) at azurecopycommand.Program.DoNormalCopy(Boolean debugMode) at azurecopycommand.Program.Main(String[] args)

On Wed, May 23, 2018 at 5:56 AM, Sonam Sunny sonam1905@gmail.com wrote:

command- azurecopy.exe -i "https:// s3.amazonaws.com/name/receipts" -o "C:\Azureifcopies" -azu rekey %AzureAccountKey% -s3k %AWSAccessKeyID% -s3sk %AWSSecretAccessKeyID% -blob copy -destblobtype block

Error log- GetHandler start GetHandler retrieved azurecopy.S3Handler GetHandler start GetHandler retrieved azurecopy.FileSystemHandler Unable to start copying receipts Exception System.ArgumentException: The argument must not be empty string. Parameter name: accountName at Microsoft.WindowsAzure.Storage.Core.Util.CommonUtility. AssertNotNullOrEmpt y(String paramName, String value) at Microsoft.WindowsAzure.Storage.Auth.StorageCredentials..ctor(String accoun tName, String keyValue, String keyName) at azurecopy.Utils.AzureHelper.GetCloudStorageAccount(String url, String acco untKey, String accountName) at azurecopy.Utils.AzureHelper.GetCloudBlobClient(String url, Boolean isSrc, String accountKey) at azurecopy.AzureBlobCopyHandler.StartCopy(BasicBlobContainer origBlob, Stri ng DestinationUrl, DestinationBlobType destBlobType, Boolean skipIfExists) at azurecopy.AzureBlobCopyHandler.StartCopyList(IEnumerable1 origBlobList, S tring destinationUrl, DestinationBlobType destBlobType, Boolean debugMode, Boole an skipIfExists) New Batch Unknown error generated. Please report to Github page https://github.com/kpfaulk ner/azurecopy/issues . Can view underlying stacktrace by adding -db flag. Syste m.ArgumentException: The argument must not be empty string. Parameter name: accountName at Microsoft.WindowsAzure.Storage.Core.Util.CommonUtility. AssertNotNullOrEmpt y(String paramName, String value) at Microsoft.WindowsAzure.Storage.Auth.StorageCredentials..ctor(String accoun tName, String keyValue, String keyName) at azurecopy.Utils.AzureHelper.GetCloudStorageAccount(String url, String acco untKey, String accountName) at azurecopy.Utils.AzureHelper.GetCloudBlobClient(String url, Boolean isSrc, String accountKey) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl, Copy Status copyStatusFilter) at azurecopy.AzureBlobCopyHandler.MonitorBlobCopy(String destinationUrl) at azurecopy.AzureBlobCopyHandler.StartCopyList(IEnumerable1 origBlobList, S tring destinationUrl, DestinationBlobType destBlobType, Boolean debugMode, Boole an skipIfExists) at azurecopycommand.Program.DoNormalCopy(Boolean debugMode) at azurecopycommand.Program.Main(String[] args) at Microsoft.WindowsAzure.Storage.Core.Util.CommonUtility. AssertNotNullOrEmpt y(String paramName, String value) at Microsoft.WindowsAzure.Storage.Auth.StorageCredentials..ctor(String accoun tName, String keyValue, String keyName) at azurecopy.Utils.AzureHelper.GetCloudStorageAccount(String url, String acco untKey, String accountName) at azurecopy.Utils.AzureHelper.GetCloudBlobClient(String url, Boolean isSrc, String accountKey) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl) at azurecopy.Utils.AzureHelper.ListBlobsInContainer(String containerUrl, Copy Status copyStatusFilter) at azurecopy.AzureBlobCopyHandler.MonitorBlobCopy(String destinationUrl) at azurecopy.AzureBlobCopyHandler.StartCopyList(IEnumerable`1 origBlobList, S tring destinationUrl, DestinationBlobType destBlobType, Boolean debugMode, Boole an skipIfExists) at azurecopycommand.Program.DoNormalCopy(Boolean debugMode) at azurecopycommand.Program.Main(String[] args)

On Tue, May 22, 2018 at 7:57 PM, Ken Faulkner notifications@github.com wrote:

Hi

Do you get any stacktrace at all?

Also, can you show me the exact command you used?

Thanks

Ken

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/kpfaulkner/azurecopy/issues/37#issuecomment-391120353, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZZYOOUo6rGoMN81ljx73rEURyIfn-9ks5t1G2VgaJpZM4UI2mF .

kpfaulkner commented 6 years ago

Hi

The command looks wrong

azurecopy.exe -i "https://s3.amazonaws.com/name/receipts" -o "C:\Azureifcopies" -azurekey %AzureAccountKey% -s3k %AWSAccessKeyID% -s3sk %AWSSecretAccessKeyID% -blobcopy -destblobtype block

The key thing here is you're telling it to use "blobcopy" which can ONLY be used if the destination is Azure (sorry if that wasn't clear). So I'm thinking it's getting a bit mixed up when telling the destination is local file but also indicating Azure is a destination. If you remove -blobcopy what are your results?

Sonam-sunny commented 6 years ago

actually i tried with both blob and local both i am getting same error

On Fri, May 25, 2018 at 3:16 AM, Ken Faulkner notifications@github.com wrote:

Hi

The command looks wrong

azurecopy.exe -i "https://s3.amazonaws.com/name/receipts" -o "C:\Azureifcopies" -azurekey %AzureAccountKey% -s3k %AWSAccessKeyID% -s3sk %AWSSecretAccessKeyID% -blobcopy -destblobtype block

The key thing here is you're telling it to use "blobcopy" which can ONLY be used if the destination is Azure (sorry if that wasn't clear). So I'm thinking it's getting a bit mixed up when telling the destination is local file but also indicating Azure is a destination. If you remove -blobcopy what are your results?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/kpfaulkner/azurecopy/issues/37#issuecomment-391875123, or mute the thread https://github.com/notifications/unsubscribe-auth/AbZZYPMp2bbkGbdU2wYWdZZ0DArWpe_Fks5t1ypSgaJpZM4UI2mF .

kpfaulkner commented 6 years ago

Can you provide the exact command you use for the Azure case (the one provided is for local filesystem). Also can you try it without -blobcopy

Thanks