[RFC] Future of NelmioApiDocBundle (v3 ?)

[joelwurtz]

Hello,

I recently contacted @willdurand for a plan about the future of this Bundle, as some of you have may have read / heard that he does not wish to maintain this libray anymore.

In this text, he speaks about Swagger v2, which, i totally agree, is now close to be a defacto standard for REST API specification and documentation.

However NelmioApiDocBundle give a very easy path for developers to expose their API documentation by just writing the server part, which brings many benefits: less maintenance, pure synchronisation between doc and code and support a large variety of API "builders" (jms, dunglas api bundle, fos rest).

Given those facts, the goal would be to keep the current easy and synchronized way of doing, but refactoring the "view" part to only expose an OpenAPI specification (new name of Swagger).

All this work will be release in a v3, we will also use this version to reduce the scope of dependencies: There will be support for Symfony >= 2.8 and PHP >= 5.5.9 (same version supported by Symfony 3.X).

There will still be work on the 2.X branch of NelmioApiDocBundle, however it will be restricted to bugs fixes at the condition it doesn't involve too many changes (it would certainly not be done if it implies to refactor too many things in order to be fix, unless there is a good PR).

Before going into this direction we would like to hear from you (people using this bundle) if you think or don't think it's a good way to improve this library.

[kix]

Keeping maintenance of this bundle is definitely a good idea. However, I'd like to add something I think NelmioApiDoc had implemented better than Swagger currently has: the sandbox. Swagger has definitions, but for some reason it lacks a way to build requests from those, while NelmioApiDoc already has this option. Perhaps, this is something Swagger should implement in their UI, but I think this was worth to note here.

[server-may-cry]

@kix, for making requests you can use external tools like postman, which can import swagger.json

[kix]

@server-may-cry, true, but, well, having this built-in makes showcasing APIs a lot easier. Add some styling and you're good to go, you can even show this API doc to a client and he can fiddle with things. Of course, Swagger-enabled clients are nice, but isn't it good to have a built-in alternative? This could make API user onboarding a lot better IMO.

[GuilhemN]

That's indeed a good idea :) I began a bundle going in that direction (but supporting many sources), maybe we can merge our efforts ?

ping @dunglas

[dunglas]

Hi everybody,

We're working hard on API Platform v2 (the first stable release is ongoing) to make it usable as a set of standalone PHP components. Some of those components are dedicated to API documentation and are a viable and modern alternative to NelmioApiDoc.

3 components can serve as a foundation for a new documentation generator:

• The "metadata" component (relying itself on Symfony's PropertyInfo component) is designed to describe the structure of a web API: resources, properties, types, validation rules... It supports raw PHP, annotations, XML and YAML: https://github.com/api-platform/core/tree/master/src/Metadata
• The "Swagger" component generates a Swagger v2 documentation using the metadata component: https://github.com/api-platform/core/tree/master/src/Swagger
• The "Hydra" component generates a Hydra documentation (and many more things related to the Hydra spec) using the metadata component: https://github.com/api-platform/core/tree/master/src/Hydra

We also integrated Swagger UI in the full stack framework to provide a nice UI similar to NelmioApiDoc out of the box.

If you want to try our Swagger implementation, the easiest way if to clone and run the last beta of the standard edition of API Platform 2: https://github.com/api-platform/api-platform/tree/v2.0.0-beta.2

@Ener-Getick started to create a bundle focused on APIDoc generation (API Platform full stack is more generalist) reusing components. The API Platform will be pleased to help and to adapt our code base to allow such usages (component based) outside of the full stack framework.

[joelwurtz]

Swagger has definitions, but for some reason it lacks a way to build requests from those

Swagger ui allows you to try request from any definition, isn't this enough ?

@dunglas i think there is room for multiple libraries generating a swagger specifiction, api platform is certainly a awesome way to build an API with Symfony, however there is still people who prefer to grap a single bundle and inject it into their current symfony project.

Also having this as a v3 will allow a smooth upgrade path to Swagger for people already using this bundle.

However, like @Ener-Getick said, i'm always in favor of sharing efforts and maybe we can find a way to work together:

• common package for serialize / deserialize a swagger spec
• Having NelmioApiDocBundle integrate to ApiPlatform
• other ideas ?

[GuilhemN]

@joelwurtz i use my own fork of gossi/swagger is that what you want ? About my bundle it is not ready to use anyway so i'm ready to deprecate it. What do you think about using it as the bases of v3?

And we can definitelly share some part of code with api platform (i'm thinking about PropertyInfo extensions, things like that :))

[joelwurtz]

I was thinking of extracting this : https://github.com/jolicode/jane-openapi/tree/master/generated into its own library, which his generated code from the swagger specification (as a json schema) so normally it has 100% support of the spec.

(We could use the generated code as a base so no need to spend too many times on it, then we update / fix manually)

[GuilhemN]

Looks good indeed :+1: would you like to create a new github team for that ?

[hpatoio]

I recently start to move away from this bundle in favour of swagger-php Of course is not as integrated as this bundle with symfony and you need a bit of extra work (many things are not taken automatically from Symfony conf/code) but it works very well.

So my point is why create another swagger generator instead of create a bridge/integration between symfony and an existing one ?

PS: I followed this path for the migration https://blog.liip.ch/archive/2016/05/11/convert-nelmioapidocbundle-to-swagger-php.html

[dunglas]

@joelwurtz Maybe I not make myself clear but I'm not speaking about using API Platform. We've done something like a "common package for serialize / deserialize a swagger spec" and we propose to make it generic enough to use it as foundation.

[GuilhemN]

@hpatoio that's more or less what i tried to do in https://github.com/EXSyst/ApiDocBundle. For example, it can read the docs generated by swagger-php but also complete it (read params from the path, read other annotations, etc).

[hpatoio]

@Ener-Getick hard to understand what that bundle was doing without doc :) Would be great if you could provide an integration/usage example !

[GuilhemN]

@hpatoio the output/definitions part is missing for now (except using swagger-php). It's a good example of what we can do but that's not ready to use in prod :)

I'll try to add a readme soon ;)

[dbu]

i like the approach of api platform with having a concept of metadata and then a separate step to generate swagger / hydra / whatever format from that. if those components can be split into separate bits that can be used stand alone (like the symfony components) that would be awesome. it would probably need an effort to reduce dependency on "core" classes of api platform - if it ends up pulling half of api platform in, i would feel not too entusiastic. and documentation and tests. which sounds like quite some effort, but could be invested at the right place.

the issues with swaggerphp are that it has no extension points where e.g. a symfony metadata reader could figure out urls and methods from the symfony route definition or the method signature, leading to a lot of redundancy. the other issue is that it comes with its own implementation of a "parser" to read the annotations from the php code, instead of using a library like doctrine annotations for that.

[dunglas]

@dbu it's designed to be split as it's done for Symfony component. Tests and dependencies are basically independents. What is missing right now is the read-only split and composer.json files for components.

[stof]

@dunglas for the split, you can contact Fabien to automate it with splitsh. He said he can run it for free for OSS projects asking for it (he is managing the split of laravel repos for instance)

[Simperfit]

We could all work together based on what we have done with the Api-Platform team for swagger.

[joelwurtz]

All this library need is a way to convert data extracted from various component into an OpenAPI (Swagger) specification, based on the previous discussion and other ideas there is ATM 4 ways of doing that:

• Using the OpenAPI (Swagger) serializer from API Platform (extracted into his own component)
• Using the OpenAPI (Swagger) serializer from gossi/Swagger (or the fork)
• Using the OpenAPI (Swagger) serializer from Jane-OpenAPI (extracted into his own library)
• Just write a simple PHP array and json_encode it

For me the first solution is not the way i would like to go, as this induce to use the internal object representation for documentation that API Platform have created to serialize into OpenAPI (Swagger) , so this would add another layer to maintain, and the goal of the v3 is to reduce the maintenance cost.

Also the first two librairies seems to lack support of certains part of the specification, like the authentication / authorization mechanism (there is a PR in the swagger one)

However Jane OpenAPI (the part i want to extract) is a pure one to one mapping of the specification, so it's not a new model it's only a object oriented view of the specification which offers better type hinting and implicit validation, also it supports 100% of the spec, and that's why i would prefer to use it.

Anyway i really don't mind having a shared repository with API Platform and/or other contributors (don't care about the vendor) who wants to have a serializer / deserializer for OpenAPI (Swagger) by giving away the code needed in Jane OpenAPI, and i think instead of using a specific serializer, maybe API Platform can use this object oriented approach when converting to an OpenAPI specification ?

The last choice may be the simplest one, but i think it will induce too many bugs due to the difficulity to type hint array values in PHP, IMO it's a no go.

[GuilhemN]

I agree with you for the metadata part of ApiPlatform but i think we can share code generating our models.

My preference goes to a part of 2 and 3 :p I mean using the generated models provided by your library as the bases and then add shortcuts on top of it (to merge documents, get parameters by their unique id (name/in), etc).

I agree with you for the last point, it looks practical but can quickly become a nightmare.

[joelwurtz]

I mean using the generated models provided by your library as the bases and then add shortcuts on top of it (to merge documents, get parameters by their unique id (name/in), etc).

I'm fine with providing shortcuts, however their should be reduce to the strict minimum, as always it add a cost on the maintenance and , IMO, this is the first measure an open source library must be aware of.

[dbu]

from my point of view, we need to separate the basic models and conversion to open-api (and possibly other formats?) from where the data for building the models comes from. the approach for building the models has to be pluggable, so that different things like configuration files, annotations or e.g. symfony extractors (and the doctrine extractors that nelmio api doc afaik has) can be combined to most efficiently build an extensive swagger doc on top of any php code.

limiting to symfony would of course make things easier, but would reduce that usability and it would be hard to split out later if its not flexible from the beginning. i did not analyse the existing implementations and how well they go with a modular approach.

it would be great if the maintainers of (some of) those 3 existing implementations could agree on a common library. if that can be any of the existing ones simply extracted or needs to be a new one, i don't know. but it would be great if we can avoid the story of:

• status there are 4 different libraries
• lets create a unified new library
• status there are 5 different libraries

[dunglas]

It should be relatively easy to transform instances of the ApiDoc annotation (the data structure storing the structure of the API in this bundle) to instances of ApiPlatform metadata's VO. It will require some adaptations to API Platform and Nelmio but when this step will be done, we got a reusable support for Swagger v2, Hydra and an UI.

[GuilhemN]

@dunglas could you provide us a small example (pseudo-code) of how you would do the conversion ? I still don't see an easy way for doing it :/

[GuilhemN]

BTW i opened https://github.com/nelmio/NelmioApiDocBundle/issues/848 a while ago but i had no answer. I applied the principles i wanted for this bundle to my own except that mine can support any library generating swagger doc.

[BigZ]

why not just create a dumper command for swaggerv2 format and deprecate the actual ui & swagger v1 dumper ? i was really kean to try dredd, so i did some quick POC and it looks like a viable way to go to me

[GuilhemN]

@BigZ because the bundle is based on an old api which is not adapted for new usages and that makes maintenance really hard.

Your example can be a short term solution but we will have to refactor the bundle or deprecate it one day anyway.

[BigZ]

yep, i suggested it to somehow keep NelmioApiDocBundle alive. But truth is, if no one wants to maintain it anymore, why not starting from the "beginning" again. It can be ok to accept multiple ways of definition, but i think we should strongly suggest one and carefully select it as for instance swagger-php introduce a lot of redundancy in a symfony application. Serializing should also be configurable as a new version of open-api standard is on the way and blueprint looks like an interesting option :)

[GuilhemN]

@BizZ my idea was to create a new bundle and support the nelmio's annotation to ease the transition to another system (or support a new system for new code).

And configurable serialization is fine for me :)

[GuilhemN]

@joelwurtz btw do you think it would be possible to support merging documents in your library ? (using the object_to_populate context key for example)

From what I see, the biggest issue is merging parameters as their unique key is not explicit.

[magnetik]

I'm willing to help (if needed) to get some of theses alternatives to move forward.

Is https://github.com/EXSyst/ApiDocBundle the current POC/Wip-project community prefered choice?

[GuilhemN]

@magnetik only me worked on it, i don't know if the others agree with this bases but imo it's a good begining.

[BigZ]

any news from the component split of API Platform @dunglas ? @GuilhemN you are the only one to have provided a draft from what i read, so if something else doesn't show up i suggest we go for your implementation idea - but for that we need a little doc first :)

[joelwurtz]

Hey, we discuss this IRL with @dunglas some weeks ago, i wanted to reported back our discussion, but just completely forget sorry :/

To resume: the main conflict point here was which representation would act as a reference for defining ressources and data from an API in the future version, as we can :

• Keep the existing one (the ApiResource annotation)
• Use the swagger as a reference by using a one to one mapping of the reference to a class system
• Use the model from API Platform

My main concern about the first one and the third one was about creating another standard for defining api ressources and data and where swagger already does the job well.

However after discussing with @dunglas it appears that the api-platform is very closed to be a one to one mapping of the Hydra specification and where all things needed by the Swagger specification should be available (at the exception of the authorization system, AFAIK).

The goal of that is to have a standard library in PHP that can represent any API working with others standards. Then if someone want to create a library on API management he can use this representation to support different API specification.

We also agree that NelmioApiDocBundle was not the right place to do such experimentation about those contracts, and that we will test this by first using the representation in this library : https://github.com/janephp/openapi which has the advantage to be smaller and allows more flexibilty due to it's young age.

Then we can bring this knowledge to NelmioDocApiBundle by providing a BC layer which will convert the internal representation to the representation provided by our work.

So yes, it's a long way before having a new version of NelmioApiDocBundle and this will certaintly not done in the next weeks / months, as i have myself not much time actually to give to open source. (i will definitely have time after christmas). But anyways there is still many issues and open pull requests that need to be taken care of before switching to this.

Also for the future version it would be preferable to have a team of people working on nelmio rather than one person (3 to 4 people would be the best).

From what i have seen @GuilhemN seems to be very interested (correct me if i'm wrong) and if there is any people wanting to help you are welcome.

What do you think about this roadmap (specially @GuilhemN given you already begin to work on it) ?

[GuilhemN]

@joelwurtz sure I'd be interested to be involved in this.

To resume: the main conflict point here was which representation would act as a reference for defining ressources and data from an API in the future version, as we can :

I think we should keep supporting nelmio's annotation but not using it as reference. I tried using @api-platform's models but as Hydra is resource-oriented while Swagger is path-oriented it was hard to pass from routes to resources. Imo it would be more efficient to use swagger based models as we could easily support Api-Platform but also Swagger-Php (some people migrated from nelmio to it) as they support swagger.

What do you think about this roadmap (specially @GuilhemN given you already begin to work on it) ?

It looks good to me. I'm not sure yet whether we should release a new version here or have a new bundle but we'll be able to discuss that later. I think we should choose a place to share our vision of the future bases of the bundle (unless you all agree to what I have done?) and then work on it as we can and we'll see when we'll be ready :)

Should we wait til christmas to begin or will you have time to discuss/review before?

[goetas]

HI all, this initiative looks very good. I'm available to help integrating jms serializer support. As I know is still the default seralization driver in fos rest bundle

[Simperfit]

@joelwurtz @GuilhemN Same I'm interested to be involved in this too just tell me ;).

[GuilhemN]

I pushed my changes to the dev branch :)

[willdurand]

@goetas @Simperfit don't hesitate to follow/review what is under development.

Closing this issue now that we have new people on board!