search

datadotworld / data-studio-connector

Google Data Studio connector for data.world
https://data.world/integrations/datastudio
Apache License 2.0
24 stars 20 forks source link

Data sampling can produce invalid JSON #6

Open rflprr opened 6 years ago

rflprr commented 6 years ago

For dataset: https://data.world/rflprr/google-ds-connector-performance

And query: SELECT * FROM hdx_org

It seems somewhere in return textLines.slice(0, sampleSize).map(JSON.parse); in rest.gs, one of the values has a starting quote but does not end the quote.

rflprr commented 6 years ago

Google App's FetchUrl service limits response size to 25Mb. Solving this issue would require: 1) Google to extend that limit for data.world, or 2) data.world to add support for HTTP range requests, so that the connector could retrieve results in 25Mb chunks