Convert JSON schema to malli

[PavlosMelissinos]

Still heavily WIP.

• [ ] Fix refs (could consider topological sorting)
• [ ] Identify missing features

Built on top of https://github.com/metosin/malli/pull/211

[timothypratley]

The simplest failing test case:

  {:$schema     "https://json-schema.org/draft/2020-12/schema",
   :$ref        "#/definitions/TopLevelSpec"
   :definitions {:A {:enum ["a" "A"]}}}
=> malli.core/invalid-schema

Some other nice small examples here: https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/main/tests/draft7/ref.json

[timothypratley]

I've been playing around with this branch and how to test it.

Adding to deps.edn test alias:

io.github.json-schema-org/JSON-Schema-Test-Suite {:git/sha       "7950d9e0579382103031ef3cfcdc37e7c58b2d1f"
                                                  :deps/manifest :deps}

Then to use it:

(def json-schema-tests-dir
  ".gitlibs/libs/io.github.json-schema-org/JSON-Schema-Test-Suite/7950d9e0579382103031ef3cfcdc37e7c58b2d1f/tests")

(def test-files
  ["draft7/ref.json"])

(defn read-test-suite-file [path]
  (-> (slurp (io/file (System/getProperty "user.home") json-schema-tests-dir path))
      (tu/from-json)
      (walk/keywordize-keys)))

(deftest json-schema-test-suite-test
  (doseq [test-file test-files]
    (testing test-file
      (doseq [{:keys [schema tests description]} (read-test-suite-file test-file)]
        (testing description
          (let [malli (try (sut/json-schema-document->malli schema)
                           (catch Throwable ex
                             (is false
                                 (str "failed to create malli for schema: " \newline
                                      schema \newline
                                      ex))))
                malli-schema (try (when malli (m/schema malli))
                                  (catch Throwable ex
                                    (is false
                                        (str "failed to create malli-schema or schema: " \newline
                                             schema \newline
                                             "from malli:" \newline
                                             malli \newline
                                             ex))))]
            (when (and malli-schema (m/schema? malli-schema))
              (doseq [{:keys [valid data description]} tests]
                (testing description
                  (is (= valid (m/validate malli-schema data))
                      (str "schema: " schema \newline
                           "data: " data \newline
                           "malli: " malli \newline
                           (if valid
                             (str "valid, but got errors: " \newline
                                  (str/join \newline (:errors (m/explain malli-schema data))))
                             "invalid, but matched"))))))))))))

Lots of things don't pass even for this one file, which is probably unsurprising as there are lots of weird edge cases in the tests.

I've been making changes to try to fix problems as I encounter them

Here is the where I got to:

parse.cljc.txt

But it still doesn't work with vega-lite

[PavlosMelissinos]

Thanks a lot for your contributions @timothypratley ! This is a weird period for me because I just moved (so I'm still settling down at home) and I haven't managed to spend time on open source at work. I'll get back on this as soon as I can though, starting with crystallizing the failing test cases you've provided into unit tests!

[ieugen]

I would have loved to have support for json-schema to use on https://github.com/compose-spec/compose-spec/blob/master/schema/compose-spec.json