alexmojaki / s3-stream-upload

Manages streaming of data to AWS S3 without knowing the size beforehand and without keeping it all in memory or writing to disk.
MIT License
208 stars 62 forks source link

Where to start #35

Closed Kalyan-D closed 3 years ago

Kalyan-D commented 3 years ago

Could you please provide us the document with a simple example on how to use this library

alexmojaki commented 3 years ago

The README links to http://alexmojaki.github.io/s3-stream-upload/javadoc/apidocs/alex/mojaki/s3upload/StreamTransferManager.html

Here is a simplified example for writing to just one stream:

AmazonS3Client client = new AmazonS3Client(awsCreds);

// Setting up
final StreamTransferManager manager = new StreamTransferManager(bucket, key, client)
        .numStreams(1)
        .numUploadThreads(2)
        .queueCapacity(2)
        .partSize(10);
MultiPartOutputStream stream = manager.getMultiPartOutputStreams().get(0);

try {
    for (int lineNum = 0; lineNum < 1000000; lineNum++) {
        String line = generateData(lineNum);

        // Writing data and potentially sending off a part
        outputStream.write(line.getBytes());
    }

    // The stream must be closed once all the data has been written
    outputStream.close();
} catch (Exception e) {

    // Aborts all uploads
    manager.abort(e);
}

// Finishing off
manager.complete();
Kalyan-D commented 3 years ago

I have an inputstream object, Could you please help us with how to use your library with the input stream object ?

Kalyan-D commented 3 years ago

We are not clear on the above example

alexmojaki commented 3 years ago

See https://github.com/alexmojaki/s3-stream-upload/issues/30 and https://github.com/alexmojaki/s3-stream-upload/issues/13