search

dekart-xyz / dekart

Create GPU-powered Kepler.gl maps from SQL queries and share them instantly and securely with live updates and multi-user editing.
https://dekart.xyz
GNU Affero General Public License v3.0
186 stars 28 forks source link

auto conversion of data types? #192

Open ttomasz opened 1 month ago

ttomasz commented 1 month ago

I have a dataset in big query table where column type is string but when loaded into dekart it becomes float. Since dataset contains strings that are not numbers (e.g. 'L702') they become NaNs and values are lost.

Does dekart try to change the data types based on some sample of data? Is there a way to override that behaviour?

delfrrr commented 1 month ago

@ttomasz hey this is kepler.gl behaviour. It guessing types by first row (from top of my head) 😔.

Common work-around is to apply sorting.

delfrrr commented 1 month ago

I will be upgrading Dekart to newer versions and evaluating moving Apache arrow which should resolve scalar type problem fundamentally.

ttomasz commented 1 month ago

@ttomasz hey this is kepler.gl behaviour. It guessing types by first row (from top of my head) 😔.

Common work-around is to apply sorting.

Wow that kind of sucks. I guess I can prepend some character in the query to the field to make it extra clear to kepler that just because something has digits does not mean it's a number. Thanks for a very fast answer.