WelliSolutions / HugeJsonViewer

Viewer for JSON files that can be GBs large.
MIT License
284 stars 18 forks source link

Treeview is slow #5

Open WelliSolutions opened 6 years ago

WelliSolutions commented 6 years ago

Thanks for the help regarding the crash. I managed to load the file now.

It's really slow to show the Treeview after load, and really slow when expanding the property that has an array of 423,828 rows.

I'm loading an Azure Table storage export, which Microsoft abused JSON to store CSV...

{"odata.metadata":"https://somewhere.table.core.windows.net/$metadata#Session",
"value":[{"PartitionKey":"0000000000000000002;0635671009200000000",
"RowKey":"m2VEe09a9TaVpVrYrQPTYO01CvqnKWrA_FiJlLaWc9Y1",
"Timestamp":"2015-05-25T16:58:46.8435944Z",
"SessionStartUtc@odata.type":"Edm.DateTime",
"SessionStartUtc":"2015-05-13T08:02:41.0000000Z"

Suspect the problem is with the TreeView which don't seem to be optimized. Committed memory is at 1,698 MB.

Are you using Virtual Mode for the TreeView?

To reduce memory usage, would it be possible to save the structure of the JSON in memory and load the actual data when it's opened? My file might be represented in this structure

{ Type: object, Offset: 0, Properties: [
    { /*odata.metadata*/Type: string, Offset: 1 }, 
    { /*value*/Type: array, Offset: 79, 
        Children: [ { Type: object, Offset: 88, Properties: [ { /*PartitionKey*/Type: string, Offset: 89 }, ...... ] } ] 
    }
]

Regards, Li Huan