TimMikeladze / next-upload

🗃️ Turn-key solution for signed & secure file-uploads to an S3 compliant storage service such as R2, AWS, or Minio. Built for Next.js. Generates signed URLs for uploading files directly to your storage service and optionally integrates with a database to store additional metadata about your files.
MIT License
99 stars 5 forks source link

400 Bad Request when using with R2 #53

Open 8times4 opened 5 months ago

8times4 commented 5 months ago

Hey Tim,

I'm getting an error when trying to use the lib with R2 for some reason. I have tested my config & endpoint using Bruno ( Postman alternative) and that works properly, but I'm getting bad request with next-upload. Have you ever tested it with R2?

Error:

image

This is my config:

export const config: NextUploadConfig = {
    maxSize: '3mb',
    bucket: '<bucket_name>',
    client: {
        region: 'auto',
        endpoint: env.R2_BUCKET_ENDPOINT,
        credentials: {
            secretAccessKey: env.R2_SECRET_KEY,
            accessKeyId: env.R2_ACCESS_KEY,
        },
    },
};

Thank you, -8x4

TimMikeladze commented 5 months ago

Let me reproduce this locally with r2. I'll get back to you in 24hrs or so.

TimMikeladze commented 5 months ago

I was able to reproduce it. I'll figure out the fix and publish it over the weekend.

TimMikeladze commented 5 months ago

Okay so my preliminary findings:

R2 does not support signed post policies (they don't mention this in the docs). I chose to use signed post policies instead of the simpler signed put urls because post policies support the conditions which provide validation of things like file size and content-types.

In order to fully support R2 I will need to implement generating signed put urls, which is totally feasible, however there may be a loss of some functionality in certain advanced edge-cases. The happy path of uploading should still work fine.

What remains is I have to figure out how to handle the maxSize option in a secure way to avoid a users forging their headers and uploading files beyond the maximum size. Hopefully R2 respects the content-length header. If it does then I will modify the logic of next-upload to have an additional step during upload which transmits the content-length of the file to the api, compares that with maxSize and then sets it as part of the headers when generating the presigned put url.

Gonna work on this over the weekend, it seems like a straight-forward implementation.

8times4 commented 5 months ago

Thank you so much for the quick turn-around! and yes, from what I've gathered, it does respect contenth-length headers :)

TimMikeladze commented 5 months ago

Update: I have R2 support working, you can see the PR here, https://github.com/TimMikeladze/next-upload/pull/58

Interesting thing I found when implementing this is that R2 needs to have a CORS config applied onto the bucket, which S3 does not.

All I have left to do is:

Will get to it in the next day or so, apologies for the delay. If you need support urgently just build a package off of my PR and it should work as long as you provide service: "r2" in the configuration object when instantiating NextUpload.

8times4 commented 5 months ago

This is awesome! Thank you so much for the quick turnaround! Will test it in a day or so :)

ringge commented 2 months ago

I also tested with BackBlaze B2, seems to have similar issue with Cloudflare R2