multiprocessio / datastation

App to easily query, script, and visualize data from every database, file, and API.
https://datastation.multiprocess.io
Other
2.9k stars 112 forks source link

Cannot create a string longer than 0x1fffffe8 characters #236

Closed nekromoff closed 2 years ago

nekromoff commented 2 years ago

[INFO] 2022-04-29T11:25:52.836Z DataStation Community Edition Panel Runner 0.9.0 DEBUG [ERROR] 2022-04-29T11:25:53.480Z Cannot create a string longer than 0x1fffffe8 characters at Buffer.toString (node:buffer:784:17) at Object.handler (/xxx/DataStation Community Edition-linux-x64/resources/app.asar/desktop/panel/columns.ts:82:3) at processTicksAndRejections (node:internal/process/task_queues:96:5) at getPanelResult (/xxx/DataStation Community Edition-linux-x64/resources/app.asar/desktop/panel/shared.ts:41:5) at evalColumns (/xxx/DataStation Community Edition-linux-x64/resources/app.asar/desktop/panel/columns.ts:50:9) at evalNoUpdate (/xxx/DataStation Community Edition-linux-x64/resources/app.asar/desktop/panel/eval.ts:301:7) at Object.handler (/xxx/DataStation Community Edition-linux-x64/resources/app.asar/desktop/panel/eval.ts:366:7) at main (/xxx/DataStation Community Edition-linux-x64/resources/app.asar/desktop/runner.ts:114:17) [INFO] 2022-04-29T11:25:53.481Z Wrote panel meta for plJdNwHGbE3P of "/xxx/DataStationProjects/test kw.dsproj" to /tmp/resultmeta--11776-xF2mrmkWZGjS.

Sample dataset I have tested on: https://eforexcel.com/wp/wp-content/uploads/2020/09/5m-Sales-Records.7z

I tried to do Pie chart visualization over data. Slice: Country Value: Total cost

eatonphil commented 2 years ago

I assume that it loaded in the file panel ok but crashed in the graph panel?

eatonphil commented 2 years ago

Ah yeah I see the limit. That's some old code with a 4gb file size limit. I'll fix for the next release. Thanks for the report!

eatonphil commented 2 years ago

Fixed in #235 you can test this out in the 0.10.0-rc1 test release if you'd like: https://github.com/multiprocessio/datastation/releases/tag/0.10.0-rc1.

Otherwise I'll post back when the next release is out.