search

akiradeveloper / akashic-storage

Filey system backed S3-compatible storage in Scala/Akka-Http
Apache License 2.0
22 stars 2 forks source link

put/get limited to 2GB #26

Closed akiradeveloper closed 8 years ago

akiradeveloper commented 8 years ago

Since netty 3 (backend of finagle-http) uses int type as the max chunk size, finch can't handle entity larger than 2GB.

finagle-http can be configured so to handle streaming entities of any size but finch at the moment doesn't support it.

There is a plan for support streaming later finch-1.0. I will wait for this update and until then sadly, akashic-storage can handle only less than 2GB objects. Wholly useless

akiradeveloper commented 8 years ago

I asked the finch's developer about workaround. Putting 2GB object is insane and multipart upload should be used instead. but Getting streaming is necessary otherwise clients can't get the uploaded object (how useless!)

If there is workaround, I will bear with that until the real implementations becomes available.

akiradeveloper commented 8 years ago

finch will support streaming response in 0.9.4, the next release!! We are able to put/get large files soon

akiradeveloper commented 8 years ago

I won't support large PUT/POST because akka-http limits the strict entity to 2GB. If I use the lazy entity as stream, larger entity can be handled but it's going to need redesign.