Stack overflow loading json schema

[Activeghost]

Using an example schema: https://github.com/ept/avrodoc/blob/master/schemata/example.avsc

I'm seeing a stack overflow with the following code:

user=>(require '[deercreeklabs.lancaster :as l]) ... define file location ... user=>(l/json->schema (slurp file-location))

I validated the avro schema used (another smaller one works fine) but regardless probably shouldn't blow the stack.

[clojure.lang.PersistentVector reduce "PersistentVector.java" 341] [clojure.core$reduce invokeStatic "core.clj" 6747] [clojure.core$reduce invoke "core.clj" 6730] [deercreeklabs.lancaster.utils$eval5666$fn__5667 invoke "utils.cljc" 1192]

... (recur until stack blown) ...

[clojure.lang.MultiFn invoke "MultiFn.java" 233] [deercreeklabs.lancaster.utils$eval5700$fn5701 invoke "utils.cljc" 1230] [clojure.lang.MultiFn invoke "MultiFn.java" 233] [deercreeklabs.lancaster.utils$eval5680$fn5681 invoke "utils.cljc" 1208] [clojure.lang.MultiFn invoke "MultiFn.java" 233] [deercreeklabs.lancaster.utils$eval5666$fn5667$fn5668 invoke "utils.cljc" 1194] [clojure.lang.PersistentVector reduce "PersistentVector.java" 341] [clojure.core$reduce invokeStatic "core.clj" 6747] [clojure.core$reduce invoke "core.clj" 6730] [deercreeklabs.lancaster.utils$eval5666$fn__5667 invoke "utils.cljc" 1192]]}

If I have time to debug this later I'll send a PR. I'm looking at using this as a starting point for:

(l/random-data schema)

functionality as a test data generator (similar to the confluent tools for their platform) but thought I'd file an issue in case I solve that problem without solving this one.

[chadharrington]

Thanks for the detailed report! I will look into this tonight.

[chadharrington]

@Activeghost Thank you again for taking the time to file this issue. It turned out to be a trivial fix. I cut a new release with the fix: version 0.6.5.

As for (l/random-data schema), I am wondering if it would be better to implement spec generation, as mentioned in this PR from @piotr-yuxuan https://github.com/deercreeklabs/lancaster/pull/3

Then, we can just generate random data using spec's generators.

What are your thoughts?

[Activeghost]

No problem, happy to bubble that up. Great to hear it was a simple fix.

I think using specs for avro serialization is an interesting idea. My upstream is writing these in avro JSON format for a Kafka connect project. So it still feels like I’d still need exposed avro -> spec functionality to get to spec -> generate -> avro -> serialize for what I’m trying to do.

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10

---

From: Chad Harrington notifications@github.com Sent: Wednesday, January 16, 2019 8:37:40 PM To: deercreeklabs/lancaster Cc: Activeghost; Mention Subject: Re: [deercreeklabs/lancaster] Stack overflow loading json schema (#4)

@Activeghosthttps://github.com/Activeghost Thank you again for taking the time to file this issue. It turned out to be a trivial fix. I cut a new release with the fix: version 0.6.5.

As for (l/random-data schema), I am wondering if it would be better to implement spec generation, as mentioned in this PR from @piotr-yuxuanhttps://github.com/piotr-yuxuan #3https://github.com/deercreeklabs/lancaster/pull/3

Then, we can just generate random data using spec's generatorshttps://clojure.org/guides/spec#_generators.

What are your thoughts?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/deercreeklabs/lancaster/issues/4#issuecomment-455039721, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKSJPaqJ3UoW8pysiPR8onKXTwdloeRzks5vD_4UgaJpZM4aChzm.