🐛 Bug Report: Premature close error

plague-doctor commented 9 months ago

👟 Reproduction steps

When downloading large file (couple of GB) the download fails after few minutes. The docker throws:

[Nest] 47  - 12/08/2023, 12:47:07 PM   ERROR [ExpressAdapter] Premature close
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
    at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
    at ServerResponse.emit (node:events:526:35)
    at emitCloseNT (node:_http_server:1023:10)
    at Socket.onServerResponseClose (node:_http_server:278:5)
    at Socket.emit (node:events:526:35)
    at TCP.<anonymous> (node:net:337:12)

Sometimes this happens several times before the file if finally downloaded.

👍 Expected behavior

Successful download regardless the file size.

👎 Actual Behavior

As described above.

🌐 Browser

Firefox, Chromium, Brave

stonith404 commented 6 months ago

@axiomen I don't have fail2ban and the error still occurs when I'm signed in :/ Did you maybe change something else?

Regarding the PayloadTooLargeError, this is fortunately easy to fix and only caused on the dev image as I have increased the chunk size for testing purposes.

axiomen commented 6 months ago

That's really bizarre. I haven't changed anything else. This is my nginx configuration file, if it helps in any way:

## Version 2023/05/31
# make sure that your pingvin-share container is named pingvin-share
# make sure that your dns has a cname set for pingvin-share

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    server_name REDACTED.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    location / {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        set $upstream_app REDACTED;
        set $upstream_port 3000;
        set $upstream_proto http;
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;

    }

}

stonith404 commented 6 months ago

The bug #401 should be resolved in the most recent 'development' image. Would you mind to try if you still don't encounter the "premature error" anymore?

axiomen commented 6 months ago

@stonith404 I'm getting 502 Bad Gateway with the latest dev image. The frontend seems to be inaccessible completely both locally and remotely. No visible error in the Pingvin logs.

stonith404 commented 6 months ago

@axiomen I pushed an new image a few minutes after I've commented. Is there maybe a newer development image than you can pull?

axiomen commented 6 months ago

@stonith404 removed and pulled the newest dev image, tested a 24GB file transfer, and haven't seen any errors. Neither the 401, nor the Premature.

babsz02 commented 6 months ago

@stonith404 Thanks for this quick version I've just tested it with an 8GB archive, several 2GB video files and no crashes. But still a very high use of RAM and even the transfer finished downloading I keep this use of RAM. I have to reboot my VM to get it back to normal. This happens to me on upload and download.

EDIT : I've just tested with another server with much more ram and I have the same RAM saturation problem.

However, when I use the free command, I don't notice any strange RAM usage.

axiomen commented 6 months ago

I'm guessing whatever you run your VM on can't tell apart free vs cached memory, so it reports that most of your memory is taken, while in reality 84% is cached (as shown with the free command). The cached part is probably the file you're downloading cached directly in the memory.

stonith404 commented 6 months ago

This should be fixed in v0.22.2. Feel free to reopen the issue if you encounter the bug again.

stonith404 commented 6 months ago

I'm reopening this as @axiomen has reported that the issue still exists.

stonith404 commented 5 months ago

This bug is really strange. @axiomen has reported that it doesn't occur anymore so I'll close it again... If you notice the error again please comment here.

sebasdt commented 5 months ago

It's just like a herobrine you try to fix It/remove the bug and it just pops up again.😜

drustan commented 4 months ago

Hi guys! I'm experiencing the same issue :

[Nest] 42  - 04/22/2024, 9:03:47 AM   ERROR [ExpressAdapter] Premature close
Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
    at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30)
    at ServerResponse.emit (node:events:530:35)
    at emitCloseNT (node:_http_server:1017:10)
    at Socket.onServerResponseClose (node:_http_server:278:5)
    at Socket.emit (node:events:530:35)
    at TCP.<anonymous> (node:net:337:12)

When I start downloading a file, the transfer is cut off after a few seconds. However, on the second attempt, it works just fine.

axiomen commented 4 months ago

@drustan on local network or remote? Is Pingvin behind a proxy?

drustan commented 4 months ago

@axiomen On a remote network, behind an Nginx reverse proxy

axiomen commented 4 months ago

The last time I had the issue, it was also on a remote network, but in another country, with most likely not 100% stable connection. It could be that Pingvin is very sensitive to losing connection, even if it is so short it's otherwise imperceivable to us. I have a Nextcloud instance on the same server, and have never experienced this issue on any connection.

drustan commented 4 months ago

The weird thing is that it's always the first try that fails, and on the second attempt, it works just fine.

axiomen commented 4 months ago

@stonith404 you should probably re-open this, unfortunately :/

hasibkyau commented 3 months ago

I am facing the same issue, and in my case, I was trying to stream pdf files. I tested with different file size and found the Premature close issue only for files whose size is above 5MB. I tested on my Windows machine along with Edge, Firefox and Chromium browsers. Followings are the test cases:

Test Case 01: File type: PDF, File Size: 1MB Result: Pass.

Test Case 02: File type: PDF, File Size: 5MB Result: Pass.

Test Case 03: File type: PDF, File Size: 10MB Result: Failed. ( ERROR [StreamableFile] Premature close)

Test Case 04: File type: PDF, File SIze: 7MB Result: Failed. ( ERROR [StreamableFile] Premature close)

MattyMay commented 2 months ago

Also having this issue. Just spent a couple hours thinking it must be the config of my reverse proxy in front of Pingvin, but now I'm not so sure.

Some details:

500MB junk file download fails 100% of the time with:
- Firefox
500MB junk file download succeeds 100% of the time with:
- Chrome
- curl
- wget
Internet connection is definitely a bit flaky, but nothing that's noticeable "during normal usage"
Cloudflare DNS with requests proxied through Cloudflare

NGINX reverse proxy sitting in front of Pingvin. Config:


server {
listen 443 ssl http2;
server_name [REDACTED];

include /etc/nginx/conf.d/snippets/ssl.conf;

set $upstream http://${PINGVIN_HOST}:${PINGVIN_PORT};

client_max_body_size      0;
proxy_buffering           off;
proxy_request_buffering   off;
sendfile                  off;

location / {
    proxy_set_header     Host $host;
    proxy_set_header     X-Real-IP $remote_addr;
    proxy_set_header     X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header     X-Forwarded-Proto $scheme;
    proxy_pass           $upstream;
}

location /robots.txt {
    add_header Content-Type text/plain;
    return 200 "User-agent: *\nDisallow: /\n";
}
}


- The error is as follows:

pingvin | {"level":"error","ts":1720589993.0274456,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","upstream":"localhost:8080","duration":0.046721392,"request":{"remote_ip":"[REDACTED]","remote_port":"39394","client_ip":"[REDACTED]","proto":"HTTP/1.0","method":"GET","host":"[REDACTED]","uri":"/api/shares/IwOTM2O/files/6b2d8478-7eba-43ef-8cbb-08a3759c75a0","headers":{"X-Real-Ip":["[REDACTED]"],"Accept-Encoding":["gzip, br"],"Accept-Language":["en-US,en;q=0.5"],"Cdn-Loop":["cloudflare"],"Upgrade-Insecure-Requests":["1"],"X-Forwarded-For":["[REDACTED]"],"Sec-Fetch-Dest":["empty"],"Cf-Ray":["8a0e35c4cd811000-LAX"],"Sec-Fetch-Site":["same-origin"],"Cf-Connecting-Ip":["[REDACTED]"],"Sec-Fetch-Mode":["same-origin"],"Priority":["u=4"],"X-Forwarded-Host":["[REDACTED]:{server_port}"],"User-Agent":["Mozilla/5.0 (X11; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"],"Referer":["[REDACTED]"],"Cf-Ipcountry":["US"],"Cookie":[],"X-Forwarded-Proto":["http"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8"],"Cf-Visitor":["{\"scheme\":\"https\"}"]}},"error":"writing: write tcp [REDACTED]->[REDACTED]: write: connection reset by peer"} pingvin | [Nest] 38 - 07/10/2024, 5:39:53 AM ERROR [StreamableFile] Premature close pingvin | Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close pingvin | at ServerResponse.onclose (node:internal/streams/end-of-stream:159:30) pingvin | at ServerResponse.emit (node:events:531:35) pingvin | at emitCloseNT (node:_http_server:1020:10) pingvin | at Socket.onServerResponseClose (node:_http_server:278:5) pingvin | at Socket.emit (node:events:531:35) pingvin | at TCP. (node:net:338:12)

stonith404 / pingvin-share