search

starburstdata / metabase-driver

Starburst Metabase driver
Apache License 2.0
60 stars 11 forks source link

Starburst json type is erroneously mapped to :type/Text #123

Open atekin opened 3 months ago

atekin commented 3 months ago

In the following code https://github.com/starburstdata/metabase-driver/blob/f4fe016552eed2567addb5c5f9af6edaa5322a59/drivers/starburst/src/metabase/driver/implementation/sync.clj#L45C40-L45C50 Starburst json type is being mapped to :type/Text. This make Metabase thinks some columns errernously as varchar and send queries like ... SUBSTRING("some_json_column", 1, 1234) "substring2494"... which throws an exception like: io.trino.spi.TrinoException: line 1:3439: Unexpected parameters (json, integer, integer) for function substring. Expected: substring(char(x), bigint), substring(char(x), bigint, bigint), substring(varchar(x), bigint), substring(varchar(x), bigint, bigint)

If one looks at postgres driver line 605 and 606 can see that https://github.com/metabase/metabase/blob/6fe41dd4379f4476b3eb78215b600393dd34474d/src/metabase/driver/postgres.clj#L605 and https://github.com/metabase/metabase/blob/6fe41dd4379f4476b3eb78215b600393dd34474d/src/metabase/driver/postgres.clj#L606

:json :type/JSON :jsonb :type/JSON

Could you please correct json fields as :type/JSON

lpoulain commented 1 month ago

@atekin I was not able to reproduce this error despite looking at a table with a JSON type. Could you provide the steps to get the error you mentioned?

atekin commented 1 month ago

I believe you need to have JSON data type, with column value length greater than 1234 characters as seen in the exception: SUBSTRING("some_json_column", 1, 1234), since it tries to get first 1234 characters. When the json column string exceeds this 1234 number it tries to get substring part of it, then trino creates the exception, since json type does not support substring function.

If you still cannot produce I can supply an example postgresql data type and values

lpoulain commented 1 month ago

I tried a large column with more than 1234 characters but couldn't reproduce the problem. And when I look at the SQL being generated by Metabase, I never see the SUBSTRING() being called. What steps in Metabase itself do you follow to generate the error? I went to "Browse data" and looked at my table.