FundingCircle / jackdaw

A Clojure library for the Apache Kafka distributed streaming platform.
https://fundingcircle.github.io/jackdaw/
BSD 3-Clause "New" or "Revised" License
369 stars 80 forks source link

Recursively defined Avro records trigger stack overflow #328

Open afhole opened 2 years ago

afhole commented 2 years ago

Avro allows records to be defined recursively. There is an example of a linked list in the Avro specification:

{
  "type": "record",
  "name": "LongList",
  "aliases": ["LinkedLongs"],                      // old name for this
  "fields" : [
    {"name": "value", "type": "long"},             // each element has a long
    {"name": "next", "type": ["null", "LongList"]} // optional next element
  ]
}

If I attempt to use a similar schema I get a stack overflow Exception: java.lang.StackOverflowError: null

I forked the repo to create a minimal reproduction of this error in the test suite: https://github.com/afhole/jackdaw/commit/8c86cc063cb86bae77fe581fb9603671342b2cc9

The results can be seen on CircleCI here: https://app.circleci.com/pipelines/github/afhole/jackdaw/2/workflows/59ba5164-4489-40fe-8e49-fb9eb5b2319c/jobs/8

Is it feasible for Jackdaw to implement this feature? Are there any alternative ways to create a serdes that does support this (e.g. interop with Apache/Confluent modules)?