Support for clojure.spec

ikitommi commented 8 years ago

Ring-swagger should support both schema.core and clojure.spec.

cprice404 commented 8 years ago

@ikitommi have you started working on this? I spent a day hacking on it and have some code that might or might not be useful depending on how far along you are.

ikitommi commented 8 years ago

About to hit vacation soon, will not have time to study/finalize this. Hacked together a small demo, where I converted the simple spec predicates into Schemas for interop, not sure that's the way to do this. My idea for the test was to reuse Schemas also for validation/coercions - much faster than spec validation and allows selecting matchers based on input (content-type etc). Another option would be to create a separate spec -> swagger json schema conversion...

Here's the one-slider of my spike: http://www.slideshare.net/metosin/schema-toolsandtricsandquickintrotoclojurespec2262016/22

If you have time, please continue/share/PR! After next thursday, I'm off, don't know how @Deraen is available in the summer time.

cheers.

cprice404 commented 8 years ago

Cool. The thing I've been hacking on is based on your second option - creating a separate spec->swagger json schema conversion. I've been using your schema->swagger conversion as a model.

My code is a mess right now but I'm hoping to get some time to clean it up a bit soon. Once it's a little cleaner I'll throw a link on this ticket. I'm working on it in a separate repo for now but would be happy to contribute it to ring-swagger if it ends up being useful.

ikitommi commented 8 years ago

Sounds great, Looking forward to it! We could release 1.0.0 after 1.9 has shipped, spec gives good excuse to clean up the public apis too.

ikitommi commented 8 years ago

I think there a three things to be specced:

1) swagger-json-schema

A good excersise on spec expressiveness. There is still the JSON Schema Validator as a real validator in ring.swagger.validator.

2) ring-swagger-schema

The expected format from downstrem libs. Currently expressed in Plumatic Schema. We could support both, in separate namespace.

3) spec as n alternative modelling language for api endpoints

Spec requires real coercions first.

lvh commented 8 years ago

I started from scratch as I happened to be using Swagger anyway and wanted to play with spec, but you raise some interesting points :) I'm mostly interested in a spec for swagger specs; I think all three of your cases are in that boat.

Starting from the JSON Schema seems like a good idea; although I suppose maybe you'll want to do codegen from that?

One thing I noticed is that because clojure.spec seems to want to use keyword names as both the identifier for the spec and the key in a map, you end up with a :swagger/swagger (for the toplevel swagger version). I don't know if there are any relevant aliasing rules...

(spec/def ::swagger
  (spec/keys :req-un [:swagger/swagger]))

(spec/def :swagger/swagger
  ;; The top-level key is called swagger. This feels dumb.
  #{"2.0"})

(spec/def :swagger/info
  (spec/keys :req-un [:swagger/api-title
                      :swagger/api-version]))

(spec/def :swagger/api-title string?)
(spec/def :swagger/api-version string?)

cprice404 commented 8 years ago

I started on something similar. I still haven't had time to clean it up and it is currently pretty tangled up with some other libraries that aren't entirely relevant. Hoping to find time to clean it up soon, but just in case any of the specs / transformation functions that I put together are useful to anyone else who is thinking about this:

https://github.com/cprice404/spec-swagger/blob/scratch/beginnings/test/cprice404/swagger_ui_service.clj#L95-L616

ikitommi commented 8 years ago

Hi guys, have you had time to play more with this? Still on vacation, so not having time to try this myself. Looking forward to your work on this! Flushing some ideas:

extra option in swagger-json to use either :schema, :spec or both. spec->swagger could be in separate ns. switch from schema to spec should be easy / setting an option for downstram libs.
when 1.9 comes out, should spec be the only way to describe the (full) swagger spec itself? Less maintenance...
would be great if use of options could be done same way as in the current schema->json-schema in multimethods &/ protocols. Stuff like :in etc. Easier for people to convert the custom schema record transformations.
still haven't figured a way to do proper coercion with spec :( This is needed to do proper runtime validation for different formats. All numbers should be conformed from strings, but just for string-type formats like query- and path-parameters. Not for things like JSON, EDN or Transit. Will try to formalize the problem to Cognitect guys... If you have brilliant ideas for this, please share :)

lvh commented 8 years ago

I haven't played more with the direct Schema specs, but I'm working on generating specs from JSON Schema, which would automagically give us Swagger. Dunno yet if that's a reasonable path to go down.

Spec definitely seems to be the "way forward" if you will -- not sure what's going on with formerly-Prismatic-now-Plumatic. It's definitely different, for now it mostly seems better-different. My main concerns are maturity and difficulty of contributing upstream.

miikka commented 8 years ago

I've started working towards generating JSON-Schemata from clojure.spec specs. The branch is here. A full integration needs quite a bit work, but it was simple enough to do something that produces sensible results for the example in README.

lvh commented 8 years ago

Whoa, that does look a lot simpler than the other direction! (Which I'm still hacking on, but it's several times longer and more complex. In particular, not having to deal with references is a godsend :D)

ikitommi commented 8 years ago

Would Spectrum help in spec->swagger? Emits Records from specs.

user=> (require '[clojure.spec :as s])
user=> (require '[spectrum.conform :as c])
user=> (c/parse-spec (s/+ integer?))
; =>
#spectrum.conform.RegexCat
{:forms [clojure.core/integer? (clojure.spec/* clojure.core/integer?)],
 :ks nil,
 :ps [#spectrum.conform.PredSpec
      {:form clojure.core/integer?, :pred #'clojure.core/integer?}
      #spectrum.conform.RegexSeq
      {:forms [clojure.core/integer?],
       :ks nil,
       :ps [#spectrum.conform.PredSpec
            {:form clojure.core/integer?, :pred #'clojure.core/integer?}],
       :ret [],
       :splice true}],
 :ret []}

ikitommi commented 8 years ago

Poked with Spectrum, looks good. Does not cover the full spec at the moment. If it would, would be really easy to do the spec -> json-schema with it. Created issue there of one missing thing. Let's see.

Also, tried to ask from the Cognitect guys could they still use Records instead of reify. Waiting for a response from Alex.

freckletonj commented 8 years ago

As far as updates, what should the adoring fans of compojure-api and ring-swagger expect going forward in regards to using clojure.spec ?

I've seen in the issues that it looks like progress is being made, especially with spec-tools, is spec officially supported yet? Or on an unreleased branch that will probably be added to master?

ikitommi commented 8 years ago

Hi, and thanks! There is no official support for spec yet and the according to the latest news, spec itself will be released some time next year. Alex hinted that there is going to be new parsing-related features in spec, might change... everything.

We are working on spec-tools right now, first release out within a week probably. what works now:

type records
dynamic/runtime conformation (string & json)
initial JSON Schema generation (but not feature complete)

needed:

more coverage & testing on the JSON Schema mappings
integrate into ring-swagger (& compojure-api)
loooot's of testing

The JSON Schema mappings & testing could use extra hands. Help welcome :)

https://github.com/metosin/spec-tools/issues/4

I have a few local spike branches of compojure-api with spec, waiting for spec-tools.

zacyang commented 7 years ago

Hi, I am looking for the tool the other way around, which could convert json schema into clojure.spec, where I end up here. Is there any tool available at the moment or spec-tool already supports that?

ikitommi commented 7 years ago

Spec-tools is out now, so spec coercion is resolved (for now, spec is still alpha, subject to change). Next: the swagger.

https://github.com/metosin/spec-tools

ikitommi commented 7 years ago

We merged spec-swagger into spec-tools. The swagger2 spec generation is now done(ish), described here: https://github.com/metosin/spec-tools/blob/master/README.md#swagger2-integration

last step to integrate with ring-swagger (&compojure-api)

ikitommi commented 7 years ago

[metosin/compojure-api "2.0.0-alpha5"] has now spec|schema + swagger - support. All ring-swagger options are not available for spec yet, will merge it later with ring-swagger. To play with the current version, there is an example repo: https://github.com/metosin/c2

ikitommi commented 6 years ago

Both the coercion and JSON Schema transformations have been ported to schema-tools & spec-tools, working also with ClojureScript now!

There is reitit (https://github.com/metosin/reitit), which has Swagger-support now for both Schema & Spec.

Might take some time to finish everything (ring-swagger would just proxy to the two underlaying libs), but I guess compojure-api and reitit can be used as examples how to integrate everything together.

Let's keep this open.

metosin / ring-swagger