search

ipfs-shipyard / py-ipfs-http-client

A python client library for the IPFS API
MIT License
682 stars 201 forks source link

client.add() stuck #245

Open zinootje opened 3 years ago

zinootje commented 3 years ago

i have a directory Data which looks like this: Data/ ├─ subfolder/ │ ├─ B.html x 5000 ├─ A.html with A about 4 mb and 5000 files B each 4kb large.

when i add it with the command line (ipfs add .\Data\ -r) it takes about 10 seconds to add. when i try to add it with you libary (client.add("Data",recursive=True)) it is still running after 5 minutes. running go-ipfs v0.7.0 localy on a windows host if you need more information let me know

zinootje commented 3 years ago

i think i found the cause , because go-ipfs sends the hash result immediately but the python http.client doesn't read the socket until all data is send it get stuck with a tcpzerowindow. This is only o probleme with a lot of files presumably because only then ipfs-go sends more data than the socket buffer

ntninja commented 3 years ago

Thank you! However, I do not believe this will be solvable using the current requests backend without resorting to bad hacks.

Will reconsider this once the new HTTPx backend is finally released.

zinootje commented 3 years ago

Your right i solved for now with some bad hacks , monky patching the request libary to read the socket every time after sending a chunk. Should a bug be filled for this with requests ?

c0llab0rat0r commented 3 years ago

Your right i solved for now with some bad hacks , monky patching the request libary to read the socket every time after sending a chunk. Should a bug be filled for this with requests ?

@zinootje,

Do you still have your monkey patch? Would you be able to share it here?

zinootje commented 2 years ago

Sorry for the late reply, do you still need it ? If so i can try to dig it up.