'connection closed before message completed' with fetch

[jgibo]

deno 1.30.3 (release, aarch64-apple-darwin) (Apple M1) v8 10.9.194.5 typescript 4.9.4

'connection closed before message completed' when using fetch, occurring sometimes locally (seemingly random), but always occurring on the 4th outbound fetch request in a Google Cloud Run First Generation service (reoccurs on 4th request when calling the endpoint again). Interestingly, in Cloud Run Second Generation the issue does not occur.

• Locally was not using Docker - running Deno directly on host

GCP article about First Generation vs Second Generation. Maybe most interestingly, 'The second generation execution environment provides full Linux compatibility rather than system call emulation.'

Will try create a reproduction sometime this week.

[aapoalas]

A reproduction, even if it only occurs sporadically (seemingly randomly), will probably be needed.

[tjconcept]

I'm seeing this too. I ported a small script from Node.js to Deno which invokes fetch on an interval. It seems the duration of the interval plays a part. 10 seconds is almost sure to hit it, but with 5 seconds it doesn't come up. I've tried isolating it to create a reproduction, but the problem disappears when I modify sufficiently.

The target server has no sign (Nginx logs) of the request.

[ttrushin]

Seeing this locally running Deno 2.0.3

[bartlomieju]

@ttrushin do you have a reproduction you could provide?

[MLB-BIOMETRIC]

I'm seeing this as well on Deno 2.0.4.

The failing function:

export async function getDeviceGroups(
    domain: string,
    csrf: string,
    sessionid: string,
) {
    const url = `https://${domain}/devicegroupdata/`;

    const options = {
        method: "GET",
        headers: {
            "Content-Type": "application/json",
            Accept: "application/json, text/plain, */*",
            "User-Agent":
                "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36",
            "X-CSRFToken": csrf,
            cookie: `sessionid=${sessionid};`,
        },
    };
    const res = await fetch(url, options);
    return await res.json() as {
        DATE_FORMAT: string;
        all_devicegroups: number[];
        totalsize: number;
        is_custom_tech: boolean;
        devicegroups: DeviceGroup[];
    };
}

[bartlomieju]

@MLB-BIOMETRIC is there a concrete domain that is giving you this error that you could share? If you're not comfortable sharing it in public we can connect over Discord to help with debugging.

[ttrushin]

@ttrushin do you have a reproduction you could provide?

Sorry about the delay, I am traveling but will try to get something over soon.

The issue occurs for me when hitting a local web server only. http://localhost:{port}/path

Adding multiple retries around the POST request is a temporary fix which works for us

[bartlomieju]

@ttrushin no worries, take your time. If you could share your server code that would be helpful too. Thanks!

[ShedFlu]

I had the same issue and was able to trace it back to concurrency. Either the API I'm currently polling (unlikely) or deno can't handle the amount of open connections. Whatever the case, I was able to fix it by only having one active request at a time. Basically keeping the fetch connection pool as empty as possible.

I don't have the time right now to find out exactly where the issue lies, but i hope this can help someone else — at least with small scripts.

[MLB-BIOMETRIC]

@bartlomieju The domain in my code above is for our enterprise solution at https://www.hexnode.com/.

The domain is like this: xxx.hexnode.com.

What is your Discord tag? We can try to debug it there.

[MLB-BIOMETRIC]

@bartlomieju Any updates on this? Should I create a better reproduction example or is the one i supplied earlier good enough?