RandomFractals / vscode-data-preview

Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files
https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.vscode-data-preview
Apache License 2.0
550 stars 59 forks source link

No result and sometimes "extension host terminated unexpectedly" with small but complex/inconsistent json file #267

Closed elder-n00b closed 3 years ago

elder-n00b commented 3 years ago

First of all, this is a minor issue in practice.

I am trying with json data from Homebrew's brew info --json=v2 --installed output. Resulting file is around 500KB.
Strangely enough, if I load the whole file there is no problem, though since it includes data for both "casks" and "formulae" (Homebrew jargon, they're sort of packages) the result isn't very useful (everything gets flattened and lumped together as "key" and "value" as expected).
So I though, I extract the two .casks and .formulae fields with jq and load those (they are both arrays but have different structure between them).
And here comes the problem: all I get is a short log and often also a message from VSCode that the "extension host has terminated unexpectedly".

Data Console Log messages

> window.WebComponentsReady
 data viewer initialized!
> document.readystatechange complete
 data.view:complete!
> loadDataInfo()
 data url: file:///Users/ [...] /formulae.json
 data table: 
> restoreConfig(
{})
> 

("data.preview.log.level": "debug" did not make a difference)

As far as I know it's valid json and jq is happy with it.
My best guess is that it is problematic because there is a lot of variability in it.
Indeed I wrote some jq to shape it in a more regular table-like form and it works beautifully with that.

Obviously vscode-data-preview expects data with some regular structure, but perhaps you may want to fall back to the key/value behavior when the schema is too convoluted (ideally, only at some level down in the hierarchy), or just throw an "invalid data" message in the Data Console Log.
Since VSCode handles the mishap with style and most users will know what they are doing, I'd say it's almost a non-issue after all, but it may be puzzling to some.

If I manage to pinpoint the problem I will update. If you need the problematic files just tell me.

versions:

P.S.: thanks, vscode-data-preview is great.

RandomFractals commented 3 years ago

not sure what your particular issue is.

data preview loads json and other data according to rules clearly documented here: https://github.com/RandomFractals/vscode-data-preview#data-preview--files-loading-details

if you are loading a json doc that doesn't contain json array, those data blobs will be shown as key/value pairs.

if you have nested json elements in your json array, they are flatten for columnar data display. see recent updates for that in releases.

you did not include any sample data or screenshots to clearly describe your issues, so I am going to close this as not reproducible.

Thanks for your feedback!

elder-n00b commented 3 years ago

lumped together as "key" and "value" as expected

If you need the problematic files just tell me.

...and here they are. I was trying to find the offending part first.

I have checked with several online validators and they're all happy.
I passed the whole "casks.json" file contents to JSON.parse() quoted with String.raw and it works.
At first I thought it was some escaping issue (lots of / and a \n in there) but wasn't that.

I think I finally got to something: arrays with mixed content, objects and strings in this case. See "test1.json" in the attached archive. Removing the two strings in the middle makes it work. "test4" and "test5" also do not load. Archive.zip

Trying the example files in data/json I noticed your extension does already warn of "invalid json data", my bad.

Also, disabling all other extensions while testing I noticed that I do not get "extension host" anymore, I now only get "loading data" stuck in the status bar (but no error message) with some of the "bad" files. Some others load successfully.

If it can't be reproduced I'll have to conclude there is something wrong on my side besides the dubious json (and I apologize in advance if that turns out to be the case).

As I said before, no big deal. Just a minor glitch with badly shaped data.
To be very clear, this is not a complaint or something, I do not pretend data-preview fixes the files for me. I just thought perhaps you may want know about this corner case.

Best regards and thanks again.

RandomFractals commented 3 years ago

Thanks for sharing your test data. I'll take a look.