search

wso2 / streaming-integrator

A stream processing runtime that allows connecting any streaming data source to any destination and act on it
Apache License 2.0
113 stars 50 forks source link

Add support for S3 multipart upload API #142

Closed sajithshn closed 2 weeks ago

sajithshn commented 4 years ago

Amazon S3 has a multipart API that lets users upload objects as multiple parts and finally merge them to form a single object. This functionality is really useful when it comes to strong event streams in (near) realtime to S3 objects.

Add support for use this API in the S3 connector (https://github.com/siddhi-io/siddhi-io-s3) of Streaming Integrator

Please refer [1] and [2] for more information

[1] - https://docs.aws.amazon.com/AmazonS3/latest/dev/usingHLmpuJava.html [2] - https://docs.aws.amazon.com/AmazonS3/latest/dev/mpListPartsJavaAPI.html

Anvesh9265 commented 8 months ago

I tried uploading a pdf into s3bukcet by sending it as encoded in request and decoding the encoded message in esb level and creating object in s3 bucket using s3 connector of 2.0.4 latest version but the file is creating with decoded content only but not in its format. so does connector allows only message data in file content field or it can also create the object with the content provided