example - serdes-kafka-avro-client.c works only for scehma with simple string

[Jagadeeshbala]

Hi, We are trying to serialize json like below: ./serdes-kafka-avro-client -P -b kafka:9092 -t test-p 1 -r http://localhost:8081 -s test -S '{"type":"record","name""test","namespace":"test","fields":[{"name":"f1","type":"string"},{"name":"f2","type":"string"}]}'

enter:str: {"f1":"value2", "f2":"value3"}

In the consumer, getting an error that its not able to deserialize it. Seems like currently with c code we are able to send only simple string format json since the json2Avro conversion is missing in the code.

/ FIXME: JSON-to-Avro conversion /

Please advise if there is any other way to get this working.

[Jagadeeshbala]

It works once I started using a custom JSON-to-Avro converter method to create the avro_value for that particular json and then passed it to the serdes_schema_serialize_avro method.

[edenhill]

@Jagadeeshbala Is this JSON-to-Avro converted something you could share?

[Jagadeeshbala]

@edenhill - For a Simple Schema like below: {"type":"record", "name":"Sample", "fields":[ {"name":"string1","type": "string","default":""}, {"name":"string2","type": "string","default":""}, {"name":"subrec1","type":{"type":"record","name":"subrec1", "fields":[ {"name":"subrec2","type":{"type":"record","name":"subrec2", "fields":[ {"name":"int1","type":"int"}, {"name":"int2","type":"int"}, {"name":"int3","type":"int"} ]}} ]}} ]}

We can use a custom converter like this and then pass the avro-value to the serdes_schema_serialize_avro method.

void json2avro(avro_value_t *p_val) { //creating avro value with these values avro_value_t field;

//string1
avro_wrapped_buffer_t  string1;
avro_wrapped_buffer_new_string(&string1, "string1");
avro_value_get_by_index(p_val, 0, &field, NULL);
avro_value_give_string_len(&field, &string1);

//string2
avro_wrapped_buffer_t string2;
avro_wrapped_buffer_new_string(&string2,"string2");
avro_value_get_by_index(p_val, 1, &field, NULL);
avro_value_give_string_len(&field, &string2);

//subrec1
avro_value_t  subrec1;
avro_value_get_by_index(p_val, 2, &subrec1, NULL);

//subrec2
avro_value_t  subrec2;
avro_value_get_by_index(&subrec1, 0, &subrec2, NULL);

//subrec2 - int1
avro_value_get_by_index(&subrec2, 0, &field, NULL);
avro_value_set_int(&field, 1);

//subrec2 - int2
avro_value_get_by_index(&subrec2, 1, &field, NULL);
avro_value_set_int(&field, 2);

//subrec2 - int3
avro_value_get_by_index(&subrec2, 2, &field, NULL);
avro_value_set_int(&field, 3);

}

Hope this helps!!

[pranayk01]

@Jagadeeshbala I tried doing what you've suggested but it's giving segmentation fault when I'm trying to produce the data. I have kept 2 fields only on which you have created the issue. Still getting the same segmentation fault issue. Can you provide more details about your implementation?

[hongbo-miao]

I provided a full demo at https://github.com/confluentinc/libserdes/issues/60 ☺️