`schema_registry_decode` fails to handle Avro schemas that nest other Avro schemas

warpstreamlabs / bento

Fancy stream processing made operationally mundane. This repository is a fork of the original project before the license was changed.

https://warpstreamlabs.github.io/bento/

Other

1.06k stars 71 forks source link

`schema_registry_decode` fails to handle Avro schemas that nest other Avro schemas #76

Closed jem-davies closed 3 months ago

jem-davies commented 4 months ago

The schema_registry_decode (and presumably schema_registry_encode) processor does not handle cases where a schema references an external schema within a nested field. This limitation affects the encoding/decoding process for Avro data that utilises such complex schemas.

Consider using confuent-kafka-go to encode/decode avro data.

jem-davies commented 3 months ago

issue replicated on branch https://github.com/warpstreamlabs/bento/tree/76-schema_registry_decode-fails-to-handle-avro-schemas-that-nest-other-avro-schemas

jem-davies commented 3 months ago

https://github.com/warpstreamlabs/bento/tree/76-schema_registry_decode-fails-to-handle-avro-schemas-that-nest-other-avro-schemas fixed on branch (needs a tidy up)