cloudflare / workers-sdk

⛅️ Home to Wrangler, the CLI for Cloudflare Workers®
https://developers.cloudflare.com/workers/
Apache License 2.0
2.58k stars 663 forks source link

🐛 BUG: R2 Multipart upload doesn't return custom metadata #5936

Open YannicEl opened 3 months ago

YannicEl commented 3 months ago

Which Cloudflare product(s) does this pertain to?

R2

What version(s) of the tool(s) are you using?

3.57.2 Wrangler

What version of Node are you using?

20.12.2

What operating system and version are you using?

Windows 11

Describe the Bug

Observed behavior

Custom metadata is missing when completing a R2 multipart upload. However when fetching the head of the r2Object the metadata is there. Strangely enough when running locally with wrangler pages dev the example works as expected. Only once you deploy your site the custom metadata is missing.

const upload = await env.BUCKET.createMultipartUpload(key, {
  customMetadata: {
    hallo: 'zwallo',
  },
});

const uploadPart = await upload.uploadPart(1, request.body);

const r2Object = await upload.complete([uploadPart]);
console.log(r2Object.customMetadata) // logs an empty object: {}

const r2Head = await env.BUCKET.head(key);
console.log(r2Head.customMetadata) // logs the correct metadata: {hallo: "zwallo"}

Expected behavior

multipartUpload.complete() should return an R2Object with the field customMetadata populated

Steps to reproduce

  1. clone repository
  2. run cd multipart-upload-missing-metadata && npm i
  3. run npm run dev for a working example
  4. run npm run deploy for the example to fail on the deployed site

Please provide a link to a minimal reproduction

https://github.com/YannicEl/cloudflare_worker_bugs/tree/main/multipart-upload-missing-metadata

Please provide any relevant error logs

No response

penalosa commented 4 weeks ago

I've pinged the R2 team, who should be able to take a look at this (cc @jonesphillip)

Schachte commented 4 weeks ago

By design, R2 does not return custom metadata on complete multipart. The "bug" is that Miniflare assumes that we do, which you can see here: https://github.com/cloudflare/workers-sdk/blob/main/packages/miniflare/src/workers/r2/bucket.worker.ts#L442

This would explain why it works locally, but not when you deploy. Locally, it's using this Miniflare instance which seems to not match R2 behavior 1:1. Our team will need to see if not returning custom metadata is by design or not internally. Might be something we overlooked, but will try and get you an answer.

Schachte commented 3 weeks ago

We discussed this and ultimately just need to prioritize plumbing this through. Will throw it on the roadmap to cleanup so we have parity with Miniflare.