search

ipfs / kubo

An IPFS implementation in Go
https://docs.ipfs.tech/how-to/command-line-quick-start/
Other
16.03k stars 3k forks source link

multiple file uploads to RPC timeout when using nginx proxy #5755

Open pbostrom opened 5 years ago

pbostrom commented 5 years ago

Version information:

go-ipfs version: 0.4.18-aefc746
Repo version: 7
System version: amd64/linux
Golang version: go1.11.1

Client: js-ipfs-api 26.1.2 running in node.

Type:

Bug

Description:

I'm having an issue uploading multiple files via the IPFS RPC when using an nginx reverse proxy. I am using the JS library js-ipfs-api in node. If I upload directly to port 5001 it works fine, but when I add the nginx proxy I get a timeout from the IPFS RPC. Ultimately my motivation is to use the nginx proxy for authentication to the IPFS RPC.

A few notes: I can upload multiple small files, but I start running into this issue when I have a couple files in the 5-7MB range. I am able to upload these files individually however.

I have created a repo to reproduce: https://github.com/pbostrom/ipfs_testcase Install docker/docker-compose and run:

git clone https://github.com/pbostrom/ipfs_testcase.git
cd ipfs_testcase
sudo docker-compose up -d
sudo docker run -it --network ipfs_testcase_proxy-tier pbostrom/ipfs_testcase

This will twice attempt to upload 3 files: the first attempt directly to the RPC service on port 5001, the second attempt will go through the nginx reverse proxy. The second attempt should stall and nginx will eventually timeout. nginx logs:

2018/11/08 18:54:09 [error] 6#6: *1 upstream timed out (110: Connection timed out) while reading upstream, client: 172.18.0.4, server: , request: "POST /api/v0/add?recursive=true&stream-channels=true HTTP/1.1", upstream: "http://172.18.0.2:5001/api/v0/add?recursive=true&stream-channels=true", host: "nginx"
172.18.0.4 - - [08/Nov/2018:18:54:09 +0000] "POST /api/v0/add?recursive=true&stream-channels=true HTTP/1.1" 200 0 "-" "/node-ipfs-api/26.1.2/"

This may be related to #5168 but I do not see the error http: invalid Read on closed Body in any of the logs or responses.

eingenito commented 5 years ago

@pbostrom thanks for the report and the awesome work to make reproducing the bug easy. I was despairing a little when I read the issue title. We'll take a look!

eingenito commented 5 years ago

So, confirmed, and also not fixed by https://github.com/ipfs/go-ipfs-cmds/pull/116. I'll look into it further.