search

nejckorasa / s3-stream-unzip

Lightweight Java library to manage unzipping of large files and data in AWS S3 without knowing the size beforehand and without keeping it all in memory or writing to disk.
https://nejckorasa.github.io/posts/s3-unzip/
MIT License
38 stars 8 forks source link

What happens if submit task fails with exception due to bounded queues & maximumPoolSize. #13

Open madhub opened 1 year ago

madhub commented 1 year ago

Hi I was trying port code to C#, during prototype I come across the situation where submitting a task fails because of bounded queues & maximumPoolSize . It looks this exception case is not handled. Executor Service can throw RejectedExecutionException Corresponding code https://github.com/nejckorasa/s3-stream-unzip/blob/b751ffd0943f1aea0e258cdae17a6ad1c246f47a/src/main/java/io/github/nejckorasa/s3/upload/S3MultipartUpload.java#L148

Suggestion: you could AWS transfer utilitie which does multiple upload of given stream under the hood.

My minimal C# implementation of the same here

https://github.com/madhub/S3StreamUnzip/blob/main/README.md