Fetching a directory listing got ridiculously slow

[RubenKelevra]

Version information:

go-ipfs version: 0.8.0-dev-895b340e0 Repo version: 10 System version: amd64/linux Golang version: go1.15.5

Description:

I'm currently testing the master prior to the 0.8 release. I noticed that the directory listing time is extremely slow. Slower than I'm remembering it. I think there was like a 10 seconds delay, but now it's minutes.

While the server I'm fetching from has 2.5 GBit/s a fast quadcore, SSD storage and more than enough memory to answer this very quickly.

Steps to reproduce:

Try to fetch a large directory listing via wget from the local ipfs daemon (you might need to add your configured port - I'm using 80).

time wget http://x86-64.archlinux.pkg.pacman.store.ipns.localhost/community/ -O /dev/null

Example

On a 8/5 MBit/s LTE connection with a latency of 30-50 ms.

[ruben@i3 ~]$ time wget http://x86-64.archlinux.pkg.pacman.store.ipns.localhost/community/ -O /dev/null
--2020-12-24 05:36:39--  http://x86-64.archlinux.pkg.pacman.store.ipns.localhost/community/
Resolving x86-64.archlinux.pkg.pacman.store.ipns.localhost (x86-64.archlinux.pkg.pacman.store.ipns.localhost)... ::1, 127.0.0.1
Connecting to x86-64.archlinux.pkg.pacman.store.ipns.localhost (x86-64.archlinux.pkg.pacman.store.ipns.localhost)|::1|:80... connected.
HTTP request sent, awaiting response... ^C

real    5m43,918s
user    0m0,002s
sys 0m0,007s

[aschmahmann]

@RubenKelevra is the report here that even when the data is stored locally a wget on the local gateway takes much longer in 0.8 than in 0.7 or 0.6, or something else? If not (i.e. the report is about a network behavior) is it reproducible in 0.8 in a way that it is not on 0.7 or a prior release?

[RubenKelevra]

@RubenKelevra is the report here that even when the data is stored locally a wget on the local gateway takes much longer in 0.8 than in 0.7 or 0.6, or something else? If not (i.e. the report is about a network behavior) is it reproducible in 0.8 in a way that it is not on 0.7 or a prior release?

I'll check that

[RubenKelevra]

So fetching the directory listing with wget takes:

uncached: 40m22,754s cached: 5m32,157s

Doing this (even cached) locks up ipfs to full CPU utilization*, while wait is barely jumping above 0%.

*(My notebook has a dual-core with hyperthreading and IPFS is using around 250%, so probably everything available)

I'll try this on a different machine

Update:

On a (low performance) server I got 0m15.389s for cached. So maybe it's a local issue on my notebook.

I try to find the issue tomorrow.

[aschmahmann]

@RubenKelevra any update on this? Is the issue still occurring reproducibly, is it new to 0.8, and any suggestions on how to reliably reproduce?

[lidel]

I believe this is resolved since 0.18: https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.18.md#-fast-directory-listings-with-dag-sizes