search

kpfaulkner / azurecopy

copy blobs between azure, s3 and local storage
Apache License 2.0
36 stars 13 forks source link

copy AWS to AzureBlob #29

Open kirubakaranja opened 7 years ago

kirubakaranja commented 7 years ago

while copy files from AWS buket to Azure Blob,

command used: azurecopy -i https://mybucket.s3-Eastus-2.amazonaws.com/buket1/ -o https://mystorage.blob.core.windows.net/mycontainereastus/sub1/sub2/sub3 -azurekey %AzureAccountKey% -s3k %AWSAccessKeyID% -s3sk %AWSSecretAccessKeyID%

Error: Unknown error generated. Please report to Github page https://github.com/kpfaulkner/azurecopy/issues . Can view underlying stacktrace by adding -db flag. azurecopy.Exceptions.CloudWriteException: AzureHandler::WriteBlob cannot write ---> System.DivideByZeroException: Attempted to divide by zero. at azurecopy.AzureHandler.ParallelWriteBlockBlob(Stream stream, CloudBlockBlob blob, Int32 parallelFactor, Int32 chunkSizeInMB) at azurecopy.AzureHandler.WriteBlob(String containerName, String blobName, Blob blob, Int32 parallelUploadFactor, Int32 chunkSizeInMB) --- End of inner exception stack trace --- at azurecopy.AzureHandler.WriteBlob(String containerName, String blobName, Blob blob, Int32 parallelUploadFactor, Int32 chunkSizeInMB) at azurecopycommand.Program.DoNormalCopy(Boolean debugMode) at azurecopycommand.Program.Main(String[] args)

kirubakaranja commented 7 years ago

above error occur: using azurecopy-1.5.0

kpfaulkner commented 7 years ago

Can you help me understand the URL structure a bit more please? You have "mybucket" at the beginning of the S3 URL (which looks correct) but then have "/buket1/" at the end of the URL. Just to confirm, is "mybucket" your REAL bucket, and "buket1" just the blob prefix you use for "virtual" directories?

Can you give me some example of your blob names?

Thanks

Ken

kirubakaranja commented 7 years ago

command used: azurecopy -i https://mybucket.s3-Eastus-2.amazonaws.com/buket1/ -o https://mystorage.blob.core.windows.net/mycontainereastus/sub1/sub2/sub3 -azurekey %AzureAccountKey% -s3k %AWSAccessKeyID% -s3sk %AWSSecretAccessKeyID%

mubucket is my bucket name & buket1 is inner folder..

even am getting same error while using whole bucket like below cmd,

azurecopy -i https://mybucket.s3-Eastus-2.amazonaws.com/ -o https://mystorage.blob.core.windows.net/mycontainereastus/sub1/sub2/sub3 -azurekey %AzureAccountKey% -s3k %AWSAccessKeyID% -s3sk %AWSSecretAccessKeyID%

kpfaulkner commented 7 years ago

Are you able to copy/paste the values inside the azurecopy.exe.config file? (obviously skipping the secrets etc). I just want to see various config items such as ChunkSizeInMB, but it might be easier if you copy the all the appsetting values (but scrubbing the secrets).

kirubakaranja commented 7 years ago

working while using... CMD: azurecopy -i https://mybucket.s3-Eastus-2.amazonaws.com/ -o https://mystorage.blob.core.windows.net/mycontainereastus/sub1/sub2/sub3 -azurekey %AzureAccountKey% -s3k %AWSAccessKeyID% -s3sk %AWSSecretAccessKeyID% -blobcopy -db

~ is there any filter (modi date value or any) avail . can you send me all parametes..

kpfaulkner commented 7 years ago

Are you saying you do not have a file called azurecopy.exe.config ? Since although you specify variables on the command line it will still read others from the file. Can I get a copy of the file?

Thanks