mgree / ffs

the file filesystem: mount semi-structured data (like JSON) as a Unix filesystem
https://mgree.github.io/ffs/
GNU General Public License v3.0
463 stars 14 forks source link

Surprisingly slow parsing #51

Open mgree opened 3 years ago

mgree commented 3 years ago

When running on the citylots JSON file, parsing takes a long, long time. Way longer than I expected---since I was told to "expect in the ballpark of 500 to 1000 megabytes per second deserialization". I suspect I'm doing something wrong.

Relatedly, lazy loading only saves time on loading, not parsing. Can we use serde_json::RawValue and similar tricks to parse but defer construction? I'm not sure every format supports this.