JedWatson commented 5 years ago

Reading through feedback on #960, I think we need to document the use-cases we're aiming at for Keystone 5, so we can more easily make decisions about api design / integrations / cli / build etc.

Might be easiest to start with an issue and then I'm happy to write up where we get to as a guide in our docs. I'm specifically focusing on web server and hosting related concerns with this issue; use-cases related to field types, graphql, etc. can be discussed and documented separately.

This also relates to #950 - we can pick major use-cases to set up as demo projects / examples, and cover the rest in guide format to document how keystone configuration scales from simple to complex requirements.

Please shout out with anything you think I'd missed, any changes to the groups and scenarios I'm suggesting, or other implications we should take into account!

NOTE Big disclaimer: I'm posting this while it's all still pretty draft. Don't assume what I'm saying below has been properly thought through or is actually best practice. This is a conversation starter so we can figure it out together!

Common Assumptions

Admin UI is optional

In each scenario below the Admin UI could be omitted entirely, leaving Keystone only providing the API, abstracting database access, and used for web server and/or common middleware features (like session authentication) where appropriate.

You'll need a database

For each scenario below you'll need a separate service to host and manage the database Keystone is connected to. This may be MongoDB Atlas, Heroku Postgres, any other cloud database provider or your own custom database server.

Other services may be necessary

Keystone includes several integrations with external services for email, image hosting, etc. The way these work wouldn't meaningfully change between different scenarios below.

Files complicate things

Local file services and static file hosting in Keystone complicate the scenarios below if they are used. To keep things simple, I'm going to leave those implications out for now.

Group 1: Integrated front-end and back-end

A) Server-side rendered website / app

Similar to the default Keystone 4 use-case. You can use Keystone as a back-end / cms for a traditional server-side generated express app (with ejs / pug templates, etc).

For this kind of setup, it mostly makes sense to host and deploy the Keystone API, Admin UI, and the app together in the same process. In particular, the app would often use Keystone's built in session management and benefit from in-process access to Keystone's list APIs.

You'd still want a build step for the Admin UI so that you're not running webpack in production, but otherwise this is concepturally similar to what Next.js provides with its dev, build and serve scripts.

With this use-case, the most common deployment target would be Heroku or a custom EC2 / linode / similar environment.

B) Universal next.js website / app (with server)

Evolution of A, you can use Keystone as a back-end / cms for a server-side and client-side universal website or app with React and Next.js. This would (I think) be consistent for Nuxt.js and other similar frameworks, but I'm less familiar with them.

This use-case brings the Admin UI and the website / app closer together in terms of architecture: in development, you want to run them all in the same process with hot reloading and debug builds; in production, you want to pre-build the source but still serve them from a node.js server which does session management, template composition / ssr, and can serve static files.

It makes sense in smaller apps for all of this to live together; as things scale, you'd want to start separating the hosting concerns so that different services were responsible for:

Static file serving (static source files, uploads, and generated assets)
HTML generation
GraphQL API

There's a whole conversation to be had around whether it's better to scale the webserver and SSR process separately from the GraphQL API or whether you benefit from in-process access to the lower-level List APIs. I think this will be up to the sophistication of the project requirements, and whoever is building the project.

For most apps, being able to access Keystone's session management and List APIs from within the Next.js app (including getInitialProps) would be a major benefit, at which point I think it makes sense to host the Admin UI and Next.js app together (at least until we've done more work around making it easy to separate these)

C) Universal next.js website / app (serverless)

Similar to B but with serverless deployment. You'd use Next.js (or a similar framework) to build for deployment to a serverless environment like Zeit's Now. This would require independent deployment of the GraphQL API(s) and Admin UI to a server environment.

This is basically the opposite of most of the things I asserted in B, where you need to separate the services from the get-go.

Future evolution of Keystone could also allow for serverless deployment of Keystone's GraphQL API(s) and Admin UI, making Keystone a more natural fit for deployment to a Now-like environment.

Group 2: Static or independent front-end

For these scenarios, Keystone's API(s) and Admin UI would be deployed and hosted completely independently from the client-side app(s) using them.

In this scenario Keystone could provide session authentication and access control around the API, and potentially function as a session service for the app or other services; however the client deployment is not affected at all by Keystone's middleware.

For some use-cases a rebuild or redeployment may be triggered by a change in state in Keystone's database or Admin UI (e.g if hosting a static blog, a build + deployment process could be triggered by changing the published status of a Post).

In other use-cases Keystone will be providing an API that the client app will communicate with remotely to query and mutate data.

D) Client-side website / app (static deployment)

With Gatsby, Next.js and Create React App (as well as similar tools for other frameworks like Vue and Angular) you could build an entirely client-side app using Keystone as a remote API.

In development, you could either combine the processes that build and serve the client-side bundles with Keystone's API and Admin UI, or manage them separately.

In production, you would build and deploy the website / app to a static host like Netlify or S3, and deploy Keystone's API and Admin UI to a Node.js server.

E) Static website

With Gatsby, Next.js and other website generators you could query Keystone's API and build static pages based on the data. This includes the ability to build and deploy static HTML that includes no run-time JavaScript at all.

You would still host Keystone in a full production environment, but that would be accessed only by content editors and the build process. The website hosting would be completely independent.

F) Mobile App

Similar to D but with stronger separation between Keystone's API and Admin UI, and the client application. Both would be built and deployed separately.

gautamsi commented 5 years ago

Similar to the default Keystone 4 use-case. You can use Keystone as a back-end / cms for a traditional server-side generated express app (with ejs / pug templates, etc).

💯

molomby commented 5 years ago

Can we clarify any of this by thinking more in terms of roles that can be fulfilled by one or more processes? .. or something? I'm guessing there are some underlying functionalities we're just moving between boxes.

molomby commented 5 years ago

Another "common assumption" we can make is that Keystone should be scaleable. As such, we should all pay close attention to the 12 factor app guidelines from Heroku:

This document synthesizes all of our experience and observations on a wide variety of software-as-a-service apps in the wild. It is a triangulation on ideal practices for app development, paying particular attention to the dynamics of the organic growth of an app over time, the dynamics of collaboration between developers working on the app’s codebase, and avoiding the cost of software erosion.

Seriously, check it out.

gabalicious commented 5 years ago

Evolution of A, you can use Keystone as a back-end / cms for a server-side and client-side universal website or app with React and Next.js.

What exactly do you mean when you say client-side universal website?

MadeByMike commented 5 years ago

@gabalicious 'Universal Applications', 'Universal JavaScript' or 'Universal Rendering' usually refers to applications that use the same JavaScript on the server as they do on the front-end. A lot of the time people are talking about server rendered React (See Next.js) or Vue (see Nuxt,js).

Just to confuse things this is sometimes called 'Polymorphic JavaScript' as well - but only by people who want to sound smart ;)