search

cloudflare / workers-sdk

⛅️ Home to Wrangler, the CLI for Cloudflare Workers®
https://developers.cloudflare.com/workers/
Apache License 2.0
2.6k stars 674 forks source link

🐛 BUG: Miniflare R2 writeHttpMetadata - DevalueError: Cannot stringify arbitrary non-POJOs #6047

Open devinpitcher opened 3 months ago

devinpitcher commented 3 months ago

Which Cloudflare product(s) does this pertain to?

R2, Miniflare

What version(s) of the tool(s) are you using?

3.60.3 [wrangler], 3.20240610.0 [miniflare]

What version of Node are you using?

18.18.2

What operating system and version are you using?

macOS Sonoma 14.4.1 (23E224)

Describe the Bug

Observed behavior

On a basic Next.js route handler, when attempting to writeHttpMetadata from an R2 Object to new Headers(), I get a DevalueError: Cannot stringify arbitrary non-POJOs error.

 ⨯ Error [DevalueError]: Cannot stringify arbitrary non-POJOs
    at flatten (file:///Users/devin/Code/xxxxxxx/node_modules/miniflare/dist/src/index.js:3035:19)
    at flatten (file:///Users/devin/Code/xxxxxxx/node_modules/miniflare/dist/src/index.js:3008:22)
    at stringify (file:///Users/devin/Code/xxxxxxx/node_modules/miniflare/dist/src/index.js:3072:17)
    at stringifyWithStreams (file:///Users/devin/Code/xxxxxxx/node_modules/miniflare/dist/src/index.js:3265:28)
    at #call (file:///Users/devin/Code/xxxxxxx/node_modules/miniflare/dist/src/index.js:6338:25)
    at Proxy.writeHttpMetadata (file:///Users/devin/Code/xxxxxxx/node_modules/miniflare/dist/src/index.js:6325:34)
    at GET (webpack-internal:///(rsc)/./src/app/api/image/[...key]/route.ts:19:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:228:37)
    at async AppRouteRouteModule.execute (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:157:26)
    at async AppRouteRouteModule.handle (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:290:30)
    at async EdgeRouteModuleWrapper.handler (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/web/edge-route-module-wrapper.js:92:21)
    at async adapter (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/web/adapter.js:179:16)
    at async (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/server/web/sandbox/sandbox.js:110:22)
    at async runWithTaggedErrors (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/server/web/sandbox/sandbox.js:107:9)
    at async DevServer.runEdgeFunction (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/server/next-server.js:1201:24)
    at async NextNodeServer.handleCatchallRenderRequest (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/server/next-server.js:248:37)
    at async DevServer.handleRequestImpl (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/server/base-server.js:816:17)
    at async (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/server/dev/next-dev-server.js:339:20)
    at async Span.traceAsyncFn (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/trace/trace.js:154:20)
    at async DevServer.handleRequest (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/server/dev/next-dev-server.js:336:24)
    at async invokeRender (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/server/lib/router-server.js:174:21)
    at async handleRequest (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/server/lib/router-server.js:353:24)
    at async requestHandlerImpl (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/server/lib/router-server.js:377:13)
    at async Server.requestListener (file:///Users/devin/Code/xxxxxxx/node_modules/next/dist/server/lib/start-server.js:141:13)

Expected behavior

The headers from the R2 object would be appended to the headers instance.

Steps to reproduce

import { type NextRequest } from "next/server";
import { getRequestContext } from "@cloudflare/next-on-pages";

export const runtime = "edge";

export async function GET(
  request: NextRequest,
  { params: { key } }: { params: { key: string[] } },
) {
  const {
    env: { UPLOADS },
  } = getRequestContext();

  const object = await UPLOADS.get(key.join("/"));

  if (object === null) {
    return new Response("Object Not Found", { status: 404 });
  }

  const headers = new Headers();

  object.writeHttpMetadata(headers);

  headers.set("etag", object.httpEtag);

  return new Response(object.body, {
    headers,
  });
}

wrangler.toml

name = "xxxxxxxxx"

compatibility_flags = ["nodejs_compat"]

[[d1_databases]]
binding = "DB"
database_name = "database"
database_id = "xxxxxxxxx"

[[r2_buckets]]
binding = "UPLOADS"
bucket_name = "xxxxxxxxx"

Please provide a link to a minimal reproduction

No response

Please provide any relevant error logs

No response

petebacondarwin commented 3 months ago

Looks like we need to make the stringification of Headers in the context of this call more robust in Miniflare. Does this code work in production?

Ken-UCD commented 3 months ago

Same error here, when uploading to a bucket. Only happens in local environment in a Next.js full-stack project.

Which Cloudflare product(s) does this pertain to?

R2, Miniflare

What version(s) of the tool(s) are you using?

3.60.3 [wrangler], 3.20240610.0 [miniflare], 1.11.3 [@cloudflare/next-on-pages]

What version of Node are you using?

20.12.2

What operating system and version are you using?

Ubuntu 22.04.3 LTS

Describe the Bug

Observed behavior

When attempting to upload through env.BUCKET.put, get [DevalueError]: Cannot stringify arbitrary non-POJOs error.

⨯ Error [DevalueError]: Cannot stringify arbitrary non-POJOs
    at flatten (file:///home/ken/dev/ai-seller/node_modules/miniflare/dist/src/index.js:3035:19)
    at flatten (file:///home/ken/dev/ai-seller/node_modules/miniflare/dist/src/index.js:3062:50)
    at flatten (file:///home/ken/dev/ai-seller/node_modules/miniflare/dist/src/index.js:3008:22)
    at stringify (file:///home/ken/dev/ai-seller/node_modules/miniflare/dist/src/index.js:3072:17)
    at stringifyWithStreams (file:///home/ken/dev/ai-seller/node_modules/miniflare/dist/src/index.js:3265:28)
    at #call (file:///home/ken/dev/ai-seller/node_modules/miniflare/dist/src/index.js:6338:25)
    at Proxy.put (file:///home/ken/dev/ai-seller/node_modules/miniflare/dist/src/index.js:6325:34)
    at POST (webpack-internal:///(rsc)/./app/api/cos/route.ts:49:116)
    at eval (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:218:43)
    at eval (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/lib/trace/tracer.js:108:36)
    at NoopContextManager.with (webpack-internal:///(rsc)/./node_modules/next/dist/compiled/@opentelemetry/api/index.js:361:30)
    at ContextAPI.with (webpack-internal:///(rsc)/./node_modules/next/dist/compiled/@opentelemetry/api/index.js:31:58)
    at NoopTracer.startActiveSpan (webpack-internal:///(rsc)/./node_modules/next/dist/compiled/@opentelemetry/api/index.js:954:34)
    at ProxyTracer.startActiveSpan (webpack-internal:///(rsc)/./node_modules/next/dist/compiled/@opentelemetry/api/index.js:994:36)
    at eval (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/lib/trace/tracer.js:97:103)
    at NoopContextManager.with (webpack-internal:///(rsc)/./node_modules/next/dist/compiled/@opentelemetry/api/index.js:361:30)
    at ContextAPI.with (webpack-internal:///(rsc)/./node_modules/next/dist/compiled/@opentelemetry/api/index.js:31:58)
    at NextTracerImpl.trace (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/lib/trace/tracer.js:97:28)
    at eval (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:206:91)
    at AsyncLocalStorage.run (node:async_hooks:346:14)
    at Object.wrap (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/async-storage/static-generation-async-storage-wrapper.js:44:24)
    at eval (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:160:288)
    at AsyncLocalStorage.run (node:async_hooks:346:14)
    at Object.wrap (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/async-storage/request-async-storage-wrapper.js:81:24)
    at eval (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:160:117)
    at AsyncLocalStorage.run (node:async_hooks:346:14)
    at AppRouteRouteModule.execute (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:157:56)
    at AppRouteRouteModule.handle (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:280:41)
    at EdgeRouteModuleWrapper.handler (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/web/edge-route-module-wrapper.js:92:44)
    at eval (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/web/adapter.js:195:23)
    at NoopContextManager.with (webpack-internal:///(rsc)/./node_modules/next/dist/compiled/@opentelemetry/api/index.js:361:30)
    at ContextAPI.with (webpack-internal:///(rsc)/./node_modules/next/dist/compiled/@opentelemetry/api/index.js:31:58)
    at NextTracerImpl.withPropagatedContext (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/lib/trace/tracer.js:63:28)
    at propagator (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/web/adapter.js:63:19)
    at adapter (webpack-internal:///(rsc)/./node_modules/next/dist/esm/server/web/adapter.js:176:22)

Example code

import type { NextRequest } from 'next/server'
import { getRequestContext } from '@cloudflare/next-on-pages';
export const runtime = 'edge'
export async function POST(req: NextRequest) {
  const object = await getRequestContext().env.BUCKET.put('test-key', req.body, {
    httpMetadata: req.headers,
  })
  return new Response(null, {headers: {'etag': object.httpEtag,}})
}

Notes

This code works fine in production. Btw, how do I access a real remote R2 in local development in a full-stack Pages project, when I am supposed to use npm run dev directly?

devinpitcher commented 3 months ago

Looks like we need to make the stringification of Headers in the context of this call more robust in Miniflare. Does this code work in production?

This is a pretty fresh project I'm working on, have not pushed to production for testing yet. I would assume it would work in production since it's just pulling a simple JPEG from R2. Must be some sort of issue with the miniflare simulator.

raffpaquin commented 3 months ago

I have the same problem with a full-stack Next.js app running locally with service binding. If my service call returns a string or number, it works as expected. But if the service call returns an object, it doesn't work, and I get the DevalueError: Cannot stringify POJOs with symbolic keys

raffpaquin commented 3 months ago

I have the same problem with a full-stack Next.js app running locally with service binding. If my service call returns a string or number, it works as expected. But if the service call returns an object, it doesn't work, and I get the DevalueError: Cannot stringify POJOs with symbolic keys

I roll-backed to wrangler@3.51 and it's now working (from wrangler@3.62).

katis commented 2 months ago

Same problem with an Astro CF Pages project with a local service binding. Latest wrangler I could get to work was wrangler@3.59

mlafeldt commented 1 month ago

I'm running into this as well.

Interestingly, if I import Headers from miniflare (like here), writeHttpMetadata works fine. Without the import, the default Headers interface from TypeScript's libdom will cause the mentioned Cannot stringify arbitrary non-POJOs error.

So this seems to be a type issue.

macinjoke commented 1 month ago

I use Rimix Vite Plugin. and same error.

Thanks to @mlafeldt , This is my workaround. 

const Headers_ =
  process.env.NODE_ENV === 'development' ? (await import('miniflare')).Headers : Headers

const headers = new Headers_() as any
object.writeHttpMetadata(headers)

In local development, 'miniflare'.Headers works. In Cloudflare Workers, normal Headers works