sodafoundation / strato

SODA Strato (Multi-cloud) project provides a cloud vendor agnostic data management for hybrid cloud, intercloud or intracloud. This project is renamed as 'Strato'
Apache License 2.0
234 stars 329 forks source link

Issue While doing Multi-part Upload on GCP (google cloud) [Just Verify] #112

Open click2cloud-alpha opened 5 years ago

click2cloud-alpha commented 5 years ago

Test Scenario: Performing migration of large data from AWS to GCP File Size : 1.0 GB File name : CentOs6.vhd Error log ss

Reason : GCP doesn’t provide Multipart Upload. Blocking : we can achieve it with GCP parallel upload but OpenSDS multi-cloud common controller architecture supports multi-part API not composite and parallel upload proposed by GCP . Limitation : GCP has limitation for multipart upload.

Reference link: https://stackoverflow.com/questions/27830432/google-cloud-storage-support-of-s3-multipart-upload https://cloud.google.com/storage/docs/composite-objects

suggested Solution : if we need to resolve this , we would need your suggestion like how we can manage this parallel and composite upload of GCP with OpenSDS Multi-cloud Controller architechture.

sfzeng commented 5 years ago

It seems gcp do support multipart upload, and our multi-cloud architecture is compatible with it, but we need more time to fix it. See: QQ图片20190419152616

skdwriting commented 5 years ago

@nguptaopensds / @sfzeng can we consider this for capri ? Please comment.

kumarashit commented 5 years ago

https://cloud.google.com/storage/docs/json_api/v1/how-tos/multipart-upload

kumarashit commented 5 years ago

https://www.zenko.io/blog/four-differences-google-amazon-s3-api/

shelar1423 commented 2 years ago

is this issue closed> ? @PravinRanjan10

PravinRanjan10 commented 2 years ago

@shelar1423 not yet. please take it and verify

merothh commented 2 years ago

WIll verify

merothh commented 2 years ago

WIll verify

Uploading a 512M file stills fails.

image