search

julia-vscode / SymbolServer.jl

Other
23 stars 31 forks source link

json storage #153

Closed ZacLN closed 3 years ago

ZacLN commented 4 years ago

I hacked together a store + read using JSON and... it's slow. 0.5-1s to convert Base from JSON to the appropriate datatypes. I'll look into a custom serialisation

davidanthoff commented 4 years ago

I guess one potential problem is that we are allocating a lot of Dicts that we then immediately throw away once we have stored stuff in the proper types, right?

Maybe also worth just running a profiler on this to see where things go wrong?

ZacLN commented 4 years ago

Yep most of the time is on the JSON side of things but I'm not really sure what can be done there tbh - any ideas?

davidanthoff commented 4 years ago

If we were to use JSON3.jl things might be faster, right? Because it skips the intermediate dict step? I guess we could use it in the LS... We would have to add three more submodules (JSON3, Parsers and StructTypes), which would be kind of annoying, but on the other hand, what the heck...

We would still have to also ship JSON.jl, for the cases where we need to have a JSON package inside a process that is owned by the user (symserver, repl, debuggee, notebook etc.), but that would probably also be ok...

Or is there some way to deserialize into structs directly with JSON.jl?

Or we go for yet another format? But that seems really not desirable to me...

ZacLN commented 4 years ago

Yeah, I'd discounted json3 as it'd be more deps. Ill have a think

davidanthoff commented 4 years ago

Should we also look at https://microsoft.github.io/language-server-protocol/specifications/lsif/0.5.0/specification/ while we are at this? My presumption is that we can't use that as our format, but maybe we should check it out?