Reading JSON representation in non-PCF

[danskarda]

Lancaster cannot read JSON schema representation which is not in Parsing Canonical Form and ignores namespace attribute.

If I understand Avro specification correctly, this should be equivalent (clojure syntax):

• {:name "foo.bar.baz"} - I believe this is called PCF
• {:name "baz", :namespace "foo.bar"}
• {:name "baz"} providing that parent has namespace foo.bar in :namespace or as a part of :name

Is it possible that JSON reader in Lancaster expects only PCF? In the following example two schemas should be equivalent. example-from-json contains :namespace, but it seems serialization and deserialization does not use it when comparing schemas.

I found this issue when playing with Confluent Schema Registry. Strangely it complains when it gets Lancaster output from json with PCF namespaces (unexpected character . in name). When you store AVRO schemas in non-PCF format with :namespace key, it does not complain but you have to preproces schema so Lancaster can ingest it.

I suggest to solve this and #25 issue together. The interesting question is how to do it in backward compatible way and not break systems which rely on current schema export to json.

(ns sandbox.avro.lancaster
  (:require [cheshire.core :as json]
            [deercreeklabs.lancaster :as avro :refer [def-record-schema]]
            [deercreeklabs.lancaster.schemas :as schema]))

(def example (schema/schema :record
                            ::test
                            [[::id :required avro/long-schema]]))

(def shared
  {:name "test"
   :namespace "sandbox.avro.lancaster"
   :type "record"
   :fields [{:name "id"
             :namespace "sandbox.avro.lancaster"
             :type "long"
             :default -1}]})

(def example-from-json
  (-> shared
      json/generate-string
      avro/json->schema))

(def example-field-from-json
  (-> shared
      (assoc :name "sandbox.avro.lancaster.test")
      json/generate-string
      avro/json->schema))

(->> (avro/serialize example {::id 1})
     (avro/deserialize example-from-json example))

;; => ExceptionInfo: Schema names does not match

(->> (avro/serialize example {::id 1})
     (avro/deserialize example-field-from-json example))

;; => {:id 1} instead of {::id 1}

[chadharrington]

Hi Dan, Thanks for this issue, as well. As you suggest, I think this can be solved alongside #25.

[chadharrington]

@danskarda I believer this issue is no longer relevant, given the discussion on #25. I will close now. If I have missed something and there is still an issue here, please let me know. Thanks!