Open lihaosky opened 2 years ago
Describe the bug When using ksql-migration tool to create table and insert values. null value was inserted for struct type.
ksql-migration
null
struct
To Reproduce Steps to reproduce the behavior, include: There are two files:
V000001__initialize_table_struct_insert_test.sql:
V000001__initialize_table_struct_insert_test.sql
CREATE TABLE struct_insert_test ( `id` VARCHAR PRIMARY KEY, `testArr` ARRAY<VARCHAR>, `testObj` STRUCT< `testAttr` VARCHAR > ) WITH ( KAFKA_TOPIC = 'struct-insert-test', VALUE_FORMAT = 'JSON', PARTITIONS = 1 );
V000002__fill_table_struct_insert_test.sql:
V000002__fill_table_struct_insert_test.sql
INSERT INTO STRUCT_INSERT_TEST ( `id`, `testArr`, `testObj` ) VALUES ( 'TEST_ID', ARRAY[ 'TEST_ITEM_1', 'TEST_ITEM_2' ], STRUCT( `testAttr` := 'TEST_VALUE' ) );
Started ksql server locally and run
bin/ksql-migrations new-project ~/ksql_migration/ http://localhost:8088
bin/ksql-migrations --config-file ~/ksql_migration/ksql-migrations.properties initialize-metadata
migrations
~/ksql_migration
./ksql-migrations -c /Users/hli/Desktop/migration_command/migrations/ksql-migrations.properties apply -n
print 'struct-insert-test' from beginning;
rowtime: 2022/11/28 19:18:14.069 Z, key: TEST_ID, value: {"testArr":["TEST_ITEM_1","TEST_ITEM_2"],"testObj":{"testAttr":null}}, partition: 0
Expected behavior key: TEST_ID, value: {"testArr":["TEST_ITEM_1","TEST_ITEM_2"],"testObj":{"testAttr":"TEST_VALUE"}}, partition: 0
key: TEST_ID, value: {"testArr":["TEST_ITEM_1","TEST_ITEM_2"],"testObj":{"testAttr":"TEST_VALUE"}}, partition: 0
Actual behaviour key: TEST_ID, value: {"testArr":["TEST_ITEM_1","TEST_ITEM_2"],"testObj":{"testAttr":null}}, partition: 0
key: TEST_ID, value: {"testArr":["TEST_ITEM_1","TEST_ITEM_2"],"testObj":{"testAttr":null}}, partition: 0
Additional context Bug is here: https://github.com/confluentinc/ksql/blob/master/ksqldb-rest-app/src/main/java/io/confluent/ksql/api/util/ApiSqlValueCoercer.java#L95-L99. The struct on line 95 is a map with key TESTATTR but the coered on line 97 has field with name testaddr. Because of the case problem, https://github.com/confluentinc/ksql/blob/master/ksqldb-rest-app/src/main/java/io/confluent/ksql/api/util/ApiSqlValueCoercer.java#L111 is false and became null.
TESTATTR
coered
testaddr
cc: @lucasbru
Describe the bug When using
ksql-migration
tool to create table and insert values.null
value was inserted forstruct
type.To Reproduce Steps to reproduce the behavior, include: There are two files:
V000001__initialize_table_struct_insert_test.sql
:V000002__fill_table_struct_insert_test.sql
:Started ksql server locally and run
bin/ksql-migrations new-project ~/ksql_migration/ http://localhost:8088
bin/ksql-migrations --config-file ~/ksql_migration/ksql-migrations.properties initialize-metadata
V000001__initialize_table_struct_insert_test.sql
andV000002__fill_table_struct_insert_test.sql
tomigrations
folder in~/ksql_migration
./ksql-migrations -c /Users/hli/Desktop/migration_command/migrations/ksql-migrations.properties apply -n
./ksql-migrations -c /Users/hli/Desktop/migration_command/migrations/ksql-migrations.properties apply -n
print 'struct-insert-test' from beginning;
from ksql cli consolerowtime: 2022/11/28 19:18:14.069 Z, key: TEST_ID, value: {"testArr":["TEST_ITEM_1","TEST_ITEM_2"],"testObj":{"testAttr":null}}, partition: 0
Expected behavior
key: TEST_ID, value: {"testArr":["TEST_ITEM_1","TEST_ITEM_2"],"testObj":{"testAttr":"TEST_VALUE"}}, partition: 0
Actual behaviour
key: TEST_ID, value: {"testArr":["TEST_ITEM_1","TEST_ITEM_2"],"testObj":{"testAttr":null}}, partition: 0
Additional context Bug is here: https://github.com/confluentinc/ksql/blob/master/ksqldb-rest-app/src/main/java/io/confluent/ksql/api/util/ApiSqlValueCoercer.java#L95-L99. The
struct
on line 95 is a map with keyTESTATTR
but thecoered
on line 97 has field with nametestaddr
. Because of the case problem, https://github.com/confluentinc/ksql/blob/master/ksqldb-rest-app/src/main/java/io/confluent/ksql/api/util/ApiSqlValueCoercer.java#L111 is false and became null.