search

confluentinc / demo-scene

👾Scripts and samples to support Confluent Demos and Talks. ⚠️Might be rough around the edges ;-) 👉For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/
https://developer.confluent.io
Apache License 2.0
1.5k stars 896 forks source link

Cast$Value Transform not working with String -> Float64? #249

Closed mmowris closed 2 years ago

mmowris commented 2 years ago

Problem

I'm using Kafka to take data from SQL Server and insert into Postgres - the issue is with billinglatitude [nvarchar] to billinglatitude [double precision].

I've tried to use Cast$Value to turn SQL Server '26.116184' to float64, but the column still comes across as a string. I can correctly cast SQL Server booleans [0, 1] to postgres [false, true].. how can I accomplish this for string -> double precision??

Attempt to Solve

I've tried to use billinglatitude:float64 and billinglatitude:float32, but the value still comes across as a string:

# Connector excerpt
  "transforms.castColumnsAccount.type": "org.apache.kafka.connect.transforms.Cast$Value",
  "transforms.castColumnsAccount.spec": "accepts_pos__c:boolean,billinglatitude:float32,billinglongitude:float64,biz_indicator__c:float64",
  "transforms.castColumnsAccount.predicates": "account",

# boolean cast, but nothing else cast...
"accepts_pos__c": false, # before cast: 0
"billinglatitude": "39.946713",
"billinglongitude": "-74.352108",
"biz_indicator__c": "2",

Reference docs

The Cast$Value documentation shows you can work with primitive types - string is a primitive type, so how can I take the string and cast it to a float??

rmoff commented 2 years ago

Hi @mmowris, you're best off asking this on the Kafka mailing list, or one of the Confluent community platforms: https://www.confluent.io/en-gb/community/ask-the-community/