nicolasdao / google-cloud-bucket

Nodejs package to manage Google Cloud Buckets and perform CRUD operations against them.
BSD 3-Clause "New" or "Revised" License
6 stars 4 forks source link

Idea: Streaming Writes #9

Open amiller-gh opened 3 years ago

amiller-gh commented 3 years ago

The module has an interface for streaming reads:

// USE CASE 3 - Piping the image buffer into a custom stream reader
const { Writable } = require('stream')
const customReader = new Writable({
    write(chunk, encoding, callback) {
        console.log('Hello chunk of image')
        callback()
    }
})
storage.get('your-bucket/a-path/image.jpg', { streamReader: customReader })
    .then(() => console.log(`Image successfully downloaded.`))

It would be great to have an interface for streaming writes too!

const fs = require('fs');
const myStream = fs.createReadStream('./my-file.json');
storage.insert(myStream, 'your-bucket/a-path/image.jpg');

Any interest in adding the feature? Happy to implement.

nicolasdao commented 3 years ago

Hi @amiller-gh,

This use case makes perfect sense. I'm currently too busy to currently implement this. I'll gladly accept your offer to help on this one. Please submit a PR whenever you're got time.

Cheers,

Nic

amiller-gh commented 3 years ago

Sounds good!

amiller-gh commented 3 years ago

Actually, since you're using node-fetch under the hood, and you just pass the payload directly to the body options... this may already work out of the box. Will test it out and let you know. I think the only problem will be with setting Content-Length to `payload.length a few lines above.

https://github.com/nicolasdao/google-cloud-bucket/blob/master/src/gcp.js#L89