Motivation

As also discussed previously, for me this is driven by wanting to be start with an OpenAPI spec, e.g. the OpenAPI spec for https://github.com/gothinkster/realworld with minimal extra extension info (https://github.com/mohawk2/SQL-Translator-Parser-OpenAPI/blob/master/t/06-corpus.json.overlay - the password thing should probably be handled with e.g. your "filter" concept on the password instead).

Then I'd be able to generate a database with SQL::Translator::Parser::OpenAPI. Then hand the spec to Yancy, and get REST API, and even web service with edit functionality.

Current shape of Yancy

My understanding of the current shape of things in Yancy is it works roughly:

backend -> bespoke-spec
     [Yancy::Backend#read_schema], goes into stash's "collections" item
[OR: manually supply bespoke-spec)
bespoke-spec -> OpenAPI
     [Mojolicious::Plugin::Yancy::_build_openapi_spec], [Yancy::Controller::Yancy::API]
bespoke-spec -> form
    [Yancy::Plugin::Form] using the stash's "collections" item

One characteristic of this is that the bespoke-spec features "x-order" values so that parameters are correctly ordered. OpenAPI uses a sequence rather than a map for specifying parameters, so using OpenAPI operations as the driver for the forms eliminates the need for this, as that information would live in the parameters spec of the mutation operation. The form could then use this instead.

Proposed new shape of working

backend -> openapi-spec
[OR: manually supply openapi-spec)
openapi-spec -> bespoke-spec # still needed?
openapi-spec -> form # reads the input parameters for the POST routes

One thing I'd like to change is from having definitions of ${name}Item and ${name}Array to just ${name}, and refer to the arrays "natively" with just the OpenAPI bit for that ({ type => 'array', items => { '$ref' => "/definitions/$name" } } doesn't feel onerous). I don't feel like having the separate definition for array adds anything. This then leaves the "definitions" quite idiomatically similar to your current bespoke-spec concept.

Parameterising forms

You've also mentioned you'd like form plugins to be able to receive a JSON schema as an input so it doesn't need to look things up from the config. Maybe that's compatible with this? Certainly I'd favour parameterising more things!

The possibility of no-code

Another thing I think would add value would be to be able to just specify a backend in the yancy.conf, also openapispec. This then offers the promise of simply being able to start up yancy with the right configuration, and an openapi spec configured in a way that JSON::Validator::OpenAPI likes (which can include URLs), without writing one's own code at all.

Conclusion

Does this make sense? Would you prefer to keep your current "shape" and enable these things via extra plugins?

preaction commented 6 years ago

I can see how what you've described here would be useful. I think the way I'd like to get there is slightly different. It's not extra plugins, really, but moving some of the current functionality into a plugin so that an author has the ability to pass in other OpenAPI specs.

As an aside, one of the reasons I haven't made the OpenAPI spec itself more accessible to modifications is that the application only supports a very narrow set of OpenAPI specs (that set being the exact OpenAPI spec produced by the plugin). My proposal below does not fix that problem, but it does open it up so that future developers could.

Right now, the relationship is more:

backend -> JSON schema -> OpenAPI Spec -> Editor application
                      \
                       -> Generated form plugin

If I rip out the "Editor application" bit into a plugin that accepts an OpenAPI spec as input, then I think that would start moving towards what you want: An editor for a specifically-organized OpenAPI spec. I say specifically-organized because there's no real way to allow arbitrary OpenAPI specs at this time (though the long-long term goal will make it easier, there's no way to get to there from where we are right now).

Yancy can already run "standalone", without code: https://metacpan.org/pod/distribution/Yancy/lib/Yancy/Help/Standalone.pod. That doc doesn't mention read_schema, because it pre-dates it. But the Yancy standalone app (Yancy.pm) could then be made to either generate an OpenAPI spec or use whatever one was passed-in.

definitions of ${name}Item and ${name}Array to just ${name}

This we should probably do regardless. It would make it easier to see the relationship between the JSON schema and the OpenAPI spec.

To move the editor into a plugin (always loaded by default, much like Mojolicious's default helpers can't be prevented from loading), the following steps would be needed:

[x] Fix definitions of ${name}Item and ${name}Array to just ${name}
- I don't remember why I did this, but if we want to support a wider range of OpenAPI specs, does it matter if they're using internal references or not?
[x] Move the building of the OpenAPI plugin into Yancy::Plugin::Editor
- Yancy::Plugin::Editor should accept a spec attribute just like Mojolicious::Plugin::OpenAPI does
[x] Move the adding of x-mojo-to configuration in the OpenAPI spec into the Yancy::Plugin::Editor
- The editor then uses Yancy::Controller::Yancy to handle communicating with the backend
- This lets us remove Yancy::Controller::Yancy::API, since it's kind of bullshit and duplicates functionality that either is or needs to be in Yancy::Controller::Yancy
- This then allows a user-generated OpenAPI spec to get the correct routes, provided we can determine what the route is trying to do
  - This is the hardest part, so we may need to ask the user to provide hints somehow
[x] Move the editor route itself into Yancy::Plugin::Editor
[x] Allow configuring the editor plugin from Mojolicious::Plugin::Yancy
- Adding an editor config to the main plugin or the standalone Yancy config file will configure Yancy::Plugin::Editor

There's still a lot of globals to deal with (the backend being a global is probably the biggest hurdle), so we'll have to work through that as we continue down this path. Does this sound like a good start towards what you're looking for?

preaction commented 5 years ago

This work is completed, though there's a bunch of work to do making the editor more extensible.

preaction / Yancy