search

confluentinc / ksql

The database purpose-built for stream processing applications.
https://ksqldb.io
Other
118 stars 1.04k forks source link

typeof scalar function - is there a built-in solution? #9480

Open sharonreytanaf opened 2 years ago

sharonreytanaf commented 2 years ago

Is there a way to get a type of value from ksqldb? for example, if the value is 123 it should return INTEGER . something like typeof or Java's getClass().getSimpleName()

suhas-satish commented 2 years ago

@sharonreytanaf , no we dont have it in ksqlDB as of today. A column already has a type. Can you use that ?

suhas-satish commented 2 years ago

are you referring to this from the Driver or the CLI?

sharonreytanaf commented 2 years ago

hey, thanks for answering and sorry for the delay.

my use case is such that i don't know the type pre-transforming because it is a mongodb cdc topic. we have clients that send us data and we don't know what they send. in mongodb it's no problem and it's saved as is, becuase it has no schema.

our sinking is to elasticsearch which does have a schema, and in this stage we need to overcome the possible schema violation. the solution is to save the field with a suffix of its type, by the value's type.

we ended up achieving that by implementing a udf. would be nice to know if there is a better way to handle such cases. i asked another question about this use case long ago in stack overflow.