cloudflare / workers-sdk

⛅️ Home to Wrangler, the CLI for Cloudflare Workers®
https://developers.cloudflare.com/workers/
Apache License 2.0
2.56k stars 659 forks source link

🐛 BUG: R2ObjectBody body `ReadableStream` not valid for put parameter #6425

Open bstopp opened 1 month ago

bstopp commented 1 month ago

Which Cloudflare product(s) does this pertain to?

R2

What version(s) of the tool(s) are you using?

3.20240725.0

What version of Node are you using?

20.13.1

What operating system and version are you using?

Mac Sonoma 14.6

Describe the Bug

Observed behavior

    const current = await env.BUCKET.get('source.html');
    const update = await env.BUCKET.put('dest.html', current.body);

Raises error: TypeError: Provided readable stream must have a known length (request/response body or readable half of FixedLengthStream)

Expected behavior

No error should be raised. Documentation indicates that body of R2ObjectBody is of type ReadableStream. Also that the put operation accepts a ReadableStream for the value.

Steps to reproduce

Please provide the following:

See Gist

Please provide a link to a minimal reproduction

https://gist.github.com/bstopp/f1f9fcdeefee94d9f1f451fa374f9273

Please provide any relevant error logs

No response

rameardo commented 1 month ago

Hi bstopp,

Can you please try this

async function readStream(stream) {
  const reader = stream.getReader();
  const decoder = new TextDecoder();
  let result = '';

  try {
    while (true) {
      const { done, value } = await reader.read();
      if (done) {
        break;
      }
      result += decoder.decode(value, { stream: true });
    }
  } catch (error) {
    console.error('Error reading stream', error);
  } finally {
    reader.releaseLock();
  }

  return result;
}

const current = await env.BUCKET.get('source.html');
if (!current.body) {
  console.log('No body found in source.html');
  return
}

const body = await readStream(current.body);
console.log('Body:', body);
const update = await env.BUCKET.put('dest.html', body);
bstopp commented 1 month ago

@rameardo - I'm not sure what benefit that solution has over what I've already implemented:

    const current = await env.BUCKET.get(key);
    const text = await current.text()
    const update = await env.BUCKET.put('other.html', text);

That it is possible to read the content into memory then, use that string as the parameter isn't the intent of this issue.

The API specifies a ReadableStream is a valid input. The body attribute of an R2ObjectBody is a ReadableStream, therefore I should be able to have the put operation perform the streaming from source to destination.

bstopp commented 1 month ago

It should be noted that i tried to pass the size of the ReadableStream via the HTTP Meatadata options:

    const key = 'index.html';
    await env.BUCKET.put(key, 'Original Content');
    const current = await env.BUCKET.get(key);
    const { size } = current;
    assert.strictEqual(size, 16);
    const update = await env.BUCKET.put('other.html', current.body, { httpMetadata: { contentLength: size } });

This also did not work.