search

denisidoro / floki

A JSON/EDN browser for the terminal
https://www.npmjs.com/package/floki
Apache License 2.0
317 stars 7 forks source link

Floki hangs on large JSON files #37

Open emiln opened 5 years ago

emiln commented 5 years ago

Floki doesn't seem able to render large JSON files at all even though JSON.parse handles them just fine.

# It's a fairly big file, sure.
ls -lh
... 31M ... big-file.json

# But node has no trouble parsing it and returning its keys.
time node -p "var fs = require('fs'); Object.keys(JSON.parse(fs.readFileSync('big-file.json')))"
[ 'totalHits', 'hitsPerPage', 'startIndex', 'hits' ]

real    0m0.422s
user    0m0.416s
sys 0m0.046s

Running floki on this file just produces a blank terminal screen even if I wait for a couple of minutes.

Do you think there's anything that can be done to improve the performance on big files? I think floki would be particularly appealing exactly in these cases of big files I'm trying to explore.

denisidoro commented 5 years ago

Regarding the parsing, I think it should be trivial to fix: the problem is that first I try to parse the input as an EDN (Clojure's JSON-like structure) then I try to parse as a JSON. I can come up with something more intelligent.

However, browsing the structure will probably remain slow because it isn't optimized: it always tries to walk all the possible paths. I would need to refractor the code a little bit.

I'll probably take a look at it next week. Thanks for pointing it out.

emiln commented 5 years ago

Thanks for looking into it. What's your stance on contributions to the project? Are you interested in pull requests for issues like this?

denisidoro commented 5 years ago

Hi! Sorry, I haven't been able to look at it yet. Any pull request is welcome! 😀

In case you try to fix this yourself I can help you out browsing the code if there's any doubt