Files 404: Improve UX when opening directory with thousands of items

[CountZer0]

Describe the bug Files reports No files here yet. Command line shows files, UI shows total files size.

To Reproduce Steps to reproduce the behavior:

1. Go to Files UI
2. See error says '404'

Expected behavior List of all the files stored.

Screenshots IPFS 404 error w/ command line proof

Desktop (please complete the following information):

• OS: Windows 10
• Browser: Chrome
• Version: ?

[welcome[bot]]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review. In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment. Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[hacdias]

Hello @CountZer0! Could you try checking if there's any errors in the developer's console? Should be accessible via CTRL-SHIFT-I or CTRL-ALT-I.

[CountZer0]

Indeed there are:

IPFS Web UI - v2.13.0 - https://github.com/ipfs-shipyard/ipfs-webui/commit/44a46ba
webui://-/locales/en-US/app.json:1 Failed to load resource: net::ERR_FILE_NOT_FOUND
webui://-/locales/en-US/welcome.json:1 Failed to load resource: net::ERR_FILE_NOT_FOUND
webui://-/locales/en-US/status.json:1 Failed to load resource: net::ERR_FILE_NOT_FOUND
webui://-/locales/en-US/files.json:1 Failed to load resource: net::ERR_FILE_NOT_FOUND
webui://-/locales/en-US/explore.json:1 Failed to load resource: net::ERR_FILE_NOT_FOUND
webui://-/locales/en-US/peers.json:1 Failed to load resource: net::ERR_FILE_NOT_FOUND
webui://-/locales/en-US/settings.json:1 Failed to load resource: net::ERR_FILE_NOT_FOUND
webui://-/locales/en-US/notify.json:1 Failed to load resource: net::ERR_FILE_NOT_FOUND
5.46be50cc.chunk.js:2 react-i18next:: It seems you are still using the old wait option, you may migrate to the new useSuspense behaviour.
r @ 5.46be50cc.chunk.js:2
DevTools failed to load source map: Could not parse content for webui://-/static/js/5.46be50cc.chunk.js.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/js/main.ab164c00.chunk.js.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/js/19.867549e7.chunk.js.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/js/1.a7a239b8.chunk.js.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/js/15.3d0e38aa.chunk.js.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/js/14.ceb748a3.chunk.js.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/js/18.8aeb256f.chunk.js.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/js/0.ab934e5a.chunk.js.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/js/13.da464701.chunk.js.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/js/9.5a744be9.chunk.js.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/css/9.d3b2790b.chunk.css.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/css/5.e3d43296.chunk.css.map: Unexpected end of JSON input
DevTools failed to load source map: Could not parse content for webui://-/static/css/main.c0cc2947.chunk.css.map: Unexpected end of JSON input

[Vectorrent]

Ditto. Exactly the same problem here: https://i.imgur.com/DZg2gJs.png

This happened immediately after I accidentally deleted a file before it had completed the IPFS import operation. The web UI is totally broken at this point.

[hacdias]

Interesting. I can't see why it would not be able to fetch the language files ad how that would influence the Files page behaviour. Could you please try opening Web UI outside of IPFS Desktop and see if you also obtain the same behaviour? What if you restart your IPFS node?

[Vectorrent]

It doesn't work in the browser, either. Restart doesn't work. Neither does a reboot or repair installation (Windows.)

[CountZer0]

^^ same

[James-E-A]

Reproducing on Ubuntu 20.04 with go-ipfs version 0.11.0 (just accessing the built-in webui with Firefox)

I really have no idea what triggered this, and the IPFS console has no errors except a bit of whining about buffer size.

[James-E-A]

Console log error that occurs after a bunch of 404:

The call to /ls?arg=QmZ…Wgi yielded a 500 server error, instead, and produced the following stacktrace:

Uncaught (in promise) HTTPError: unexpected EOF
    Babel 3
        o
        t
        exports
    n error.js:24
    e core.js:91
    l runtime.js:63
    _invoke runtime.js:293
    x runtime.js:118
    Babel 6
        n
        s
        promise callback*n
        s
        exports
        exports
    O core.js:73
    e http.js:147
    l runtime.js:63
    _invoke runtime.js:293
    x runtime.js:118
    Babel 6
        n
        s
        promise callback*n
        s
        exports
        exports
    value 5.46be50cc.chunk.js:2
    fetch core.js:192
    value http.js:175
    n ls.js:49
    l runtime.js:63
    _invoke runtime.js:293
    x runtime.js:118
    Babel 4
        i
        _invoke
        _invoke
        next
    e index.js:9
    l runtime.js:63
    _invoke runtime.js:293
    x runtime.js:118
    Babel 4
        n
        s
        exports
        exports
    a index.js:9
    e actions.js:623
    l runtime.js:63
    _invoke runtime.js:293
    x runtime.js:118
    Babel 2
        r
        s
construct.js:19:20
    Babel 6
        r
        u
        (Async: promise callback)
    r
        s
        i
        i
    doFilesFetch actions.js:188
    Redux 4
        c
        u
        g
        action
    e index.js:49
    default FilesPage.js:49
    React 2
        us
        Su
    unstable_runWithPriority scheduler.production.min.js:19
    React 3
        qi
        xu
        su
    su self-hosted:1225
    Ji React
    unstable_runWithPriority scheduler.production.min.js:19
    React 6
        qi
        Ji
        Xi
        nu
        enqueueSetState
        setState
    <anonymous> self-hosted:1178
    G Redux
    forEach self-hosted:4353
    G Redux

[James-E-A]

Is the common factor here "users of the IPFS-companion add-on"?

[James-E-A]

I really have no idea what triggered this

Update: This was caused (in my case) by putting non-UnixFS data into MFS:

ipfs files cp /ipfs/QmPS4nWyVQwdPye6EeDzabPjVj32ZX34qPz1dGGuwBHiTz /mathjax@3.2.0.tar

required field "Type" not set

Note that this only breaks the view of the particular directory it's in — if you, for instance, visit #/files/ipfs-companion-imports, that should still work even if your root view is broken

---

Workaround/Fix

#d="/some-subfolder" # OPTIONAL; leave unset to look at MFS root

(IFS='
'; for f in $(ipfs files ls "$d/"); do
  ipfs files stat "$d/$f" --format "$d/$f OK (<hash>)" || (
    printf 'BROKEN PATH FOUND: %s\n' "$d/$f"
    printf "TO REMOVE IT, RUN:\n\t%s\n" "ipfs files cp /ipfs/bafkreihdwdcefgh4dqkjv67uzcmw7ojee6xedzdetojuzjevtenxquvyku \"$d/$f\" && ipfs files rm \"$d/$f\""
  )
done)

[SgtPooki]

Test dir CID with 10k images: bafybeiggvykl7skb2ndlmacg2k5modvudocffxjesexlod2pfvg5yhwrqm fast dir listing (skips size resolution: ipfs ls -s --size=false --resolve-type=false $CID