honojs / hono

Web framework built on Web Standards
https://hono.dev
MIT License
19.06k stars 543 forks source link

TypeError: Response body object should not be disturbed or locked #1695

Open RobertSasak opened 10 months ago

RobertSasak commented 10 months ago

What version of Hono are you using?

3.9.0

What runtime/platform is your app running on?

Firebase functions

What steps can reproduce the bug?

Here is a simple example with POST end point.

import { getRequestListener } from '@hono/node-server'
import { Hono } from 'hono'

const app = new Hono()

app.post('/hello', async (c) => {
  return c.text('Hello World!')
})

export const web = onRequest(getRequestListener(app.fetch))

What is the expected behavior?

Requst should reach Heno and Hello World should appear.

What do you see instead?

There is an following silent error 500 when doing a POST request with payload. By placing an console.log in https://github.com/honojs/node-server/blob/3575d60a15e345700b81e22376825e98b0a69ebe/src/listener.ts#L42 it is possible to see this error.

TypeError: Response body object should not be disturbed or locked
    at extractBody (node:internal/deps/undici/undici:6433:17)
    at new Request (node:internal/deps/undici/undici:7314:48)
    at /<path>/functions/node_modules/@hono/node-server/dist/listener.js:48:33
    at /<path>/functions/lib/functions/src/index.js:79:64
    at /<path>/functions/node_modules/firebase-functions/lib/v2/providers/https.js:57:29
    at cors (/<path>/functions/node_modules/cors/lib/index.js:188:7)
    at /<path>/functions/node_modules/cors/lib/index.js:224:17
    at originCallback (/<path>/functions/node_modules/cors/lib/index.js:214:15)
    at /<path>/functions/node_modules/cors/lib/index.js:219:13
    at optionsCallback (/<path>/functions/node_modules/cors/lib/index.js:199:9)

Additional information

Similar issue is documented here https://github.com/honojs/node-server/issues/84

As far as I understand the issue occurs in https://github.com/honojs/node-server/blob/3575d60a15e345700b81e22376825e98b0a69ebe/src/listener.ts#L40 . Here the new Request is created from incoming request.

Firebase use expressjs to process all request and also expose Express JS Request object as an incoming. However this request is already "processed" and the actual body stream is already read.

So when on line https://github.com/honojs/node-server/blob/3575d60a15e345700b81e22376825e98b0a69ebe/src/listener.ts#L38 the read stream is create there is nothing to read.

I am not sure if this is an issue of Hono or how to approach it.

My workaround is to patch listner file and restore body from rawBody.

diff --git a/node_modules/@hono/node-server/dist/listener.js b/node_modules/@hono/node-server/dist/listener.js
index 3f5eaf7..e3541d0 100644
--- a/node_modules/@hono/node-server/dist/listener.js
+++ b/node_modules/@hono/node-server/dist/listener.js
@@ -38,7 +38,14 @@ const getRequestListener = (fetchCallback) => {
       headers: headerRecord
     };
     if (!(method === "GET" || method === "HEAD")) {
-      init.body = import_node_stream.Readable.toWeb(incoming);
+      // init.body = import_node_stream.Readable.toWeb(incoming);
+      init.body = new ReadableStream({
+        start(controller){
+          controller.enqueue(incoming.rawBody);
+          // controller.enqueue('{"c":"d"}');
+            controller.close();
+        }
+      });
       init.duplex = "half";
     }
     let res;
yusukebe commented 10 months ago

Hi @RobertSasak,

Thank you for creating this. Since it's an issue with @hono/node-server (it's fine to create the issue here), could you first provide the version number of your @hono/node-server?

RobertSasak commented 10 months ago

Originaly I tested on 1.2.0. I have updated to 1.2.2 just now and the issue still occurs.

AverageHelper commented 10 months ago

I'm having this issue as well. Seems to have to do with that Readable.toWeb call.

Tested on Node 18.17.1, hono 3.10.2, @hono/node-server 1.2.3.

AverageHelper commented 10 months ago

The above workaround doesn't seem to work in my case. The Request gets constructed just fine, but when my handler calls c.req.json(), a TypeError: Received non-Uint8Array chunk is thrown.

ZanzyTHEbar commented 8 months ago

any head-way on this?

HananoshikaYomaru commented 8 months ago

c.req.json() and c.req.valid("json") don't work and make it almost unusable.

noidwasavailable commented 8 months ago

Is there an update to this?

Looks like this might be related to: https://github.com/vercel/next.js/issues/50166 and https://github.com/FirebaseExtended/firebase-framework-tools/pull/122

cogoo commented 6 months ago

I had this identical issue with using vercel funcitons.

I found a solution that might help you further debug the problem.

A similar issue highlights the code in the vercel dev serve that causese the issue: https://github.com/honojs/node-server/issues/84#issuecomment-1844850121

The issue can be resolved by setting the following environment variable in your local environment file:

NODEJS_HELPERS="0"

It might be worth checking if Firebase Functions has similar behaviour

yusukebe commented 5 months ago

Hi. Is this problem still happening? I don't have an environment for Firebase. If it's not resolved, I'll make it and try to investigate.

remorses commented 4 months ago

This problem still happens when you cancel a request that is in progress

Bobetti commented 2 months ago

has the exact same error using npm - 10.5.0, "vercel": "^35.2.3", "hono": "^4.5.1", "@hono/node-server": "^1.12.0", Switched over to bun, no such error anymore.

Bobetti commented 1 month ago

using npm i vercel@latest and "@hono/node-server/vercel" - any request to the hono server ends up with image

Error occurs if request contains payload, in my case simple json.

@yusukebe Is it hono problem or vercel problem?

NODEJS_HELPERS=0 not helping

Bobetti commented 1 month ago

I also noticed that if i turn off cors middleware, and the post route looks like this:

.post("/pricing", async (c) => {      

        // const body = await c.req.json();

        return c.json({ success: true });
    });

Everything works.

But as soon as I turn on cors - I get the error: "Response body object should not be disturbed or locked". If I turn off cors but uncomment the above line const body = await c.req.json(); i get the same error.

Bobetti commented 1 month ago

@yusukebe As I see, no responses in this thread. Does it mean, that HonoJs is just not compatible with Vercel's node js environment? And there is no point in waiting for the fast solution and better to move to another framework?

yusukebe commented 1 month ago

@Bobetti

Sorry for the lack of response. If you can share a minimal repo to reproduce it, I can investigate.

Bobetti commented 1 month ago

@yusukebe Here is the minimal project: https://www.dropbox.com/scl/fi/4qwy7ckiql8uiu8cyr4ot/hono-vercel.zip?rlkey=trq71faz162w5zbpk2rptb6q2&dl=0

I removed vercel info and node modules.

Request I'm sending from svelte application using this code (using hono RPC):

const makeRequest = async () => {

        console.log('makeRequest');

        try {
            const dataClient = hc<ApiPricingRoutes>('/');
            const api = dataClient.api;
            const res = await api.pricing.$post({
                json: {
                    countryCode: 'Estonia',
                    startDateUtc: '2024-07-25T21:00:00.000Z',
                    endDateUtc: '2024-07-26T20:59:59.999Z'
                }
            });

            if (!res.ok) {
                const errorText = 'Request failed: ' + res.statusText;
                toast.error(errorText);
                return;
            }

            const result = await res.json();
            toast.success('Request successful: ');
        } catch (error) {
            console.error('There was a problem with the fetch operation:', error);
        }
    };

Thank you very much for your help @yusukebe

P.S.

Error itself:

image

yusukebe commented 1 month ago

@Bobetti

I've tried it, but I can't understand how to reproduce it. And this project is not minimal. I'm sorry, but I can't help you.

Bobetti commented 1 month ago

@yusukebe What does it mean you can't reproduce it? I tried it using macbook M2 air, I tried it using windows 10 machine. Always the same result. I even installed Thunder Client in VS Code to make requests without svelte client. And the result is always the same: image

Is it really working for you? Did you use vercel node.js, see after nr 4, as written here: https://hono.dev/docs/getting-started/vercel

Exactly the same request in Thunder client, working perfectly using NestJS.

May be you need some more data?

yusukebe commented 1 month ago

Hi @Bobetti

Sorry, it is reproduced. This is a Vercel matter. The stream of IncomingMessage from Vercel has already been used. The reason for this is that incoming.complete is true. I thought we could prevent it with the option NODEJS_HELPERS=0, but it does not affect it. This is Vercel matter, so it's difficult to fix it in Hono-side.

Bobetti commented 1 month ago

@yusukebe thank you very much for your help.

maxweisel commented 1 week ago

We're hitting this same issue (we do not use vercel or other middleware that adds helpers). It seems specific to sockets getting closed while the request handler is still trying to fulfill the request. Specifically canceling existing requests leads to this bug:

node:internal/deps/undici/undici:5312
          throw new TypeError(
          ^

TypeError: Response body object should not be disturbed or locked
    at extractBody (node:internal/deps/undici/undici:5312:17)
    at new Request (node:internal/deps/undici/undici:9492:48)
    at new Request (file:///usr/src/app/node_modules/@hono/node-server/dist/index.mjs:29:5)
    at newRequestFromIncoming (file:///usr/src/app/node_modules/@hono/node-server/dist/index.mjs:60:10)
    at [getRequestCache] (file:///usr/src/app/node_modules/@hono/node-server/dist/index.mjs:81:35)
    at [getAbortController] (file:///usr/src/app/node_modules/@hono/node-server/dist/index.mjs:76:26)
    at ServerResponse.<anonymous> (file:///usr/src/app/node_modules/@hono/node-server/dist/index.mjs:404:34)
    at ServerResponse.emit (node:events:519:28)
    at emitCloseNT (node:_http_server:1020:10)
    at TLSSocket.onServerResponseClose (node:_http_server:278:5)

This is running in docker on linux.

yusukebe commented 1 week ago

Hi @maxweisel

Can you provide a minimal project to reproduce it? It's better it work on my MacOS machine(not on docker) if possible.

ivank commented 2 days ago

Also experiencing this when functions are deployed to google functions.

import { getRequestListener } from '@hono/node-server';
import { http } from '@google-cloud/functions-framework';
import { app } from './app';

http('handler', getRequestListener(app.fetch));

With the following code, any post is not working when deployed to google functions (gen2). I think it is the way google "prefetches" the data for the functions rather than streaming it in.