[UX] New documentation hierarchy suggestion

bpolaszek commented 5 years ago

Hey there,

API-Platform is an ecosystem full of features and tools, but its documentation is growing fast, and it begins to lack of an appropriate structure to keep things tidy and clear.

I have identified several drawbacks on the current documentation that could lead astray newcomers:

The Api Component sublevels are far too many, and don't share the same level of importance (i.e NelmioApiDocBundle being sibling of Operations, for instance)
Starting by the Distribution makes look like Api-Platform can't work without a proper Docker setup and all the things (mercure, h2 push, varnish, let's encrypt, postgresql, ...), which isn't true. With the advent of Flex, everything is done in Symfony to make applications from simple to advanced: you only composer req the components you need. Beside, if we can assume the user has a minimal knowledge of Symfony (it is used under the hood along with Doctrine, share the same concepts, components and structure), Api-Platform can work without Docker, so we shouldn't add Docker into the learning curve if it's not mandatory. As a consequence, I think the distribution should not be the very starting point, but exposed as a great tool to get a full-featured environment. The starting point could be a simple composer req api && bin/console server:start, to focus on the API itself instead of its environment. In the meantime, example commands should not be prefixed by docker-compose exec.
Similarly, some components (schema generator, client generator, ...) are useful tools but may not deserve a top-level position in the documentation hierarchy (generators being useful doesn't mean they are necessary). It make them look like they're completely part of Api-Platform and you can't work without them, which is wrong. At least it makes you focus on the stack instead of the core. They could be grouped in a sub level and/or expanded in a separate documentation.
Some topics cover easy setup as well as advanced coding and configuration. While this might be a good idea at first glance, most of the time it leads to extremely scrollable pages and things the user doesn't want to read at this moment. It also makes the information hard to find when you're looking for something. Hacks and tricks should be moved away in a specific section, in my opinion (and linked from the main topics they're related to).
The documentation sections hierarchy should target newcomers at first, and go to experienced users at the end.
An F.A.Q. or How do I? section is really missing. Having such a section could also lead some developers to share their own recipes and help the community.

Read -> Install -> Configure -> Play -> Code -> Test -> Deploy -> Tune -> Hack

After thinking about this, I suggest the following key topics to drive the user from basics to advanced:

API-Platform

Introduction

State of the art, philosophy, features
Installation

On an existing project / new project / docker distribution
Usage and configuration

Schema generator, entity map, serialization, events, ...
Pagination, filters and sorting

You can start to play with your data
Security

JWT, FOSUserBundle, ACLs
Exposing your API

Versioning, deprecations, sunset, error messages, ...
Consuming your API

OpenAPI/Swagger export, GraphQL support, Client generators
The Admin Component

Big topic that should be kept handled apart
Testing

Fixtures, writing functional tests
Deployment

Kubernetes, Heroku, Platform.sh, ...
Caching & performance optimization

Cache invalidation, custom headers, Doctrine tips, PHP-PM, Blackfire
Advanced Usage

Custom operations/controllers, DTOs, custom filters, data providers/persisters, CQRS, ...
How do I?

Enable MongoDB, ElasticSearch, NelmioApiDocBundle, Mercure, file uploads, ...
Troubleshooting

Docker issues, Nginx issues, JMS serializer issues, links to SF Slack / Stackoverflow

A more detailed reorganization is available on this gist.

What do you think?

Sorry for the long post! 😅

alanpoulain commented 5 years ago

I think the aim was the contrary: having the distribution to be the common way to use API Platform and the best setup to present the examples. But your point is very good and maybe we should reconsider it. I don't think a lot of people use the distribution as is.

You're perfectly right about restructuring the documentation. API Platform is growing and the current documentation is messy: I think new developers are quickly lost and maybe some people give up their motivation for using it when they see the documentation.

(and I think GraphQL should have its own topic :stuck_out_tongue:)

bpolaszek commented 5 years ago

Hey @alanpoulain, thanks for your review.

+1 for GraphQL, maybe between The Admin Component and Testing ?

alanpoulain commented 5 years ago

IMHO before The Admin Component would be better since the admin could be compatible with GraphQL as well.

dunglas commented 5 years ago

I'm 👍 to restructure the documentation, and most ideas looks good to me but:

API Platform is a full stack framework. And is not (only) a PHP framework. JS tools will even be promoted more with the upcoming release of Admin 1.0 and the support of Next, Quasar etc in client generator. The next big milestone is OpenAPI support for those tools, to be able to use them with any API (even non-API Platform/Hydra ones). Those component must be promoted on the homepage, in the menu etc.
Docker improves the experience a lot. Installing API Platform (and even Symfony) without Docker is a pain (I do a lot of workshops, I trained hundreds of newcomers, getting started with Docker is a matter of minutes, and to run a single command, setting up a PHP + Postgres environment is difficult and literally takes hours). Benefiting of all the features provided by API Platform requires more setup (Node for the JS tools, Mercure, soon Vulcain...). So the distribution will stay the recommended entry point, allowing to use all features with ease. composer require is supported, and already mentioned in the Getting Started guide for advanced users. And I don't want to rely on the symfony binary for API Platform because (unlike Docker) it is not a free software.
We do not target (only) Symfony developers. We must not assume that newcomers have Symfony (and even PHP) skills. It's often not the case (many React/Vue/Angular developers use API Platform because it is better than most API frameworks written in JS and very easy to use even if you don't know Symfony). Of course to do advanced skills with API Platform, Symfony skills are required (it's why we link to the appropriate pages on Symfony.com), but it's not a prerequisite to use the framework.

Also, there was a bug on the website affecting the order in the menu (the order was totally irrelevant because of this bug). It has been fixed by https://github.com/api-platform/website/pull/148 3 days ago. The order should looks more logical now 😅, but it can definitely be improved.

dunglas commented 5 years ago

I like the new TOC by the way, for the Core component.

I would suggest some changes regarding the it:

The Getting Started / Quick Start must stay first (I expect a a doc to create a quick project before the design considerations, philosophy etc)
Data Providers and Persisters must be more prominent because they are very important concepts (see also #867). What about a section "Fetching Persisting Data" with data providers and persisters, Doctrine ORM, Mongo and Elastic docs?
GraphQL must have its dedicated, prominent, section "REST and GraphQL framework to build modern API-driven projects"
OpenAPI, Hydra, Swagger UI, Redoc, GraphiQL etc could be under a "docs & specs" section
Mercure (and at some point maybe other related topics such as Websockets, Kafka...) should have its own top level section "Real-time and streaming capabilities"

And! The "advanced" section is a very very good idea!!

dunglas commented 5 years ago

Regarding the docker-compose exec... prefix. I don't like it either. We should explain to the user how to create a bin/console alias executing docker-compose exec php bin/console under the hood the use bin/console everywhere.

alanpoulain commented 5 years ago

I don't really agree with this part:

Docker improves the experience a lot. Installing API Platform (and even Symfony) without Docker is a pain.

You're right when the user wants to try it, in a workshop or to show it, but when someone or a company wants to really use it in their project, they don't need (and don't want) to use the full framework with all this stuff in it. They want to pick what they need and it's often a composer req api. And I think we need to target these users in the documentation. That's why Symfony Flex was created: real users want to explicitly choose what they use.

Of course, I also think promoting the full ecosystem is a great thing to do, to envision all the possibilities. But we should show it off in the main page and maybe in the first part of the "Getting Started" topic but the advanced documentation should not use it.

dunglas commented 5 years ago

Not using Docker for a professional project is not an option in 2019 IMHO. And users wanting to customize everything can do it, it’s documented (but I don’t think it’s a best practice). Most users should use Docker and it’s what we must promote.

alanpoulain commented 5 years ago

Yes, but I think, most of the time, users have already their own Docker images and want to use API Platform in existing projects. Users starting from scratch by using the distribution images should not represent a lot of people (but it's hard to measure).

bpolaszek commented 5 years ago

Regarding Docker:

You're right when the user wants to try it, in a workshop or to show it, but when someone or a company wants to really use it in their project, they don't need (and don't want) to use the full framework with all this stuff in it. They want to pick what they need and it's often a composer req api. And I think we need to target these users in the documentation. That's why Symfony Flex was created: real users want to explicitly choose what they use.

Of course, I also think promoting the full ecosystem is a great thing to do, to envision all the possibilities. But we should show it off in the main page and maybe in the first part of the "Getting Started" topic but the advanced documentation should not use it.

Totally agree with @alanpoulain.

I don't know any PHP developer who doesn't have php/composer installed locally on his system (i.e. out of docker). Is it a real pain to run composer create-project symfony/skeleton my_api && cd my_api && composer req api? Setting up a MariaDb/SQLite DB takes seconds on any UNIX system (if it's not already done) and serving the API for demo purposes is as simple as composer req --dev server && bin/console server:start. I'm not even talking about the Symfony binary: WebServerBundle is MIT-licensed. You won't have HTTPS, H2 push, Mercure or Varnish but it's way enough, as a beginner, to get started with config, pagination, filtering, access controls etc.
Not using Docker for a professional project is not an option in 2019 IMHO: I disagree either. Promoting it is a good idea, but enforcing it is a wrong one. It's like Symfony would mix PHPStorm's Symfony shortcuts in all their documentation: PHPStorm will surely enhance DX a lot (and I personnally can't do anything professionnal without it), but it can't be a first-level requirement. The Distribution is an awesome package to show the whole API-Platform ecosystem power as an overview in a near-production environment, but it looks like a lot of boilerplate code that won't obviously match the end-user requirements. When doing workshops and trainings, if you want to show Api-P along with Varnish, Mercure, Postgre etc, the Distribution is definitely the way to go. But If you only focus on the core, is all that stuff really needed? apt/brew/snap/pkg/yum install php mysql and downloading composer won't take you more than 2 minutes (having postgre, different versions of PHP etc is cool, but it's a totally different topic). Besides, not all IT-teams are devops-compliant. You can perfectly run a versionned project, deployed with Deployer/Capistrano/whatever with 0-downtime on a load-balanced architecture with master/slaves DB, in a professional way, without having to know anything about Docker, because sysadmins know their job and developers know their own. Not following today's fashion doesn't mean "unprofessional". You shouldn't judge your users: they can be as big as Facebook or Amazon, and as little as a guy wanting to run an API-first todolist for his family on a $5 VPS in a few minutes; that's the philosophy of Symfony, AFAIK. TL;DR: the whole ecosystem is great, but I think we should focus on what matters first: exposing an API, consuming this API.

Regarding your following comment:

API Platform is a full stack framework. And is not (only) a PHP framework. JS tools will even be promoted more with the upcoming release of Admin 1.0 and the support of Next, Quasar etc in client generator. The next big milestone is OpenAPI support for those tools, to be able to use them with any API (even non-API Platform/Hydra ones). Those component must be promoted on the homepage, in the menu etc.

I understand (and share) your point, but it actually sounds confusing. JS tools are part of the stack but they're designed to work as standalones. When invoking the API-Platform name, what comes to your mind? The full-stack framework? The core? The Docker Distribution? A set of standalone components designed to work together? Maybe all this could be clarified, not at the documentation level, but at the website one (with icons, separate menus, etc). Maybe in a second time. Anyway, as they're designed to work as standalone components, I agree they deserve a 1st place in the overall documentation.

Regarding your suggested changes in the core documentation: I agree with all of them, I just have a remark about Data Provider/Data persisters: I understand it is an essential part of the core, but I definitely consider them as an advanced user concern. Understanding the core's usability is totally feasible without them, because of the great Doctrine support, but when you want to encapsulate a 3rd-party API or use Solr or any unsupported storage, you enter the "hack" section, IMO. I did not put this section as the 1st sublevel of Advanced usage because I thought that you're more likely to need DTOs or custom operations than replacing Doctrine, but I'm open to another POV.

alanpoulain commented 5 years ago

I thought that you're more likely to need DTOs or custom operations than replacing Doctrine

Using custom DataProvider/Persister is also very useful when you want to have a resource not linked to Doctrine. Typically when you need to create a "command" endpoint, for sending a mail for instance or when it impacts multiple entities and not just one. @teohhanhui has used this pattern a lot I think for the Sylius Shop API (which is not a CRUD one).

bpolaszek commented 5 years ago

@alanpoulain Interesting, I didn't think of doing this that way. Is there a public example of this? This kind of tricks would be very welcome in the new How do I? subject :-)

alanpoulain commented 5 years ago

I think @dunglas tries to explain it here: https://github.com/api-platform/docs/pull/888. You can see this pattern here (for resetting a password): https://github.com/dunglas/ShopApiPlugin/blob/api-platform/src/Command/Customer/ResetPassword.php https://github.com/dunglas/ShopApiPlugin/tree/api-platform/src/DataPersister https://github.com/dunglas/ShopApiPlugin/blob/api-platform/src/Handler/Customer/ResetPasswordHandler.php

bpolaszek commented 5 years ago

Another recurrent consistency issue:

Disabling Swagger UI

# api/config/packages/api_platform.yaml
api_platform:
    # ...
    enable_swagger_ui: false

It will switch your default documentation UI to ReDoc.

Manually Registering the Swagger UI Controller

# app/config/routes.yaml
swagger_ui:
    path: /docs
    controller: api_platform.swagger.action.ui

api/config/packages/api_platform.yaml refers to the distribution path
app/config/routes.yaml refers to a Symfony 3 configuration

My suggestion:

config/packages/api_platform.yaml
config/routes.yaml

WDYT?

api-platform / docs