Research what it would take to migrate the backend to Prisma

[wdoug]

Prisma is the successor to the graphcool-framework backend that we are using currently. Previously we avoided this because we would have to set up a new database and jump through several other hoops.

It is possible this is still the case, but the benefits might be worth the cost. One thing to note is that running:

npm install -g graphql-cli
graphql create my-app --boilerplate node-basic

(see https://github.com/graphql-boilerplates/node-graphql-server/tree/master/basic) will automatically set up a database and a live graphql server for you, so maybe we can just use that functionality (I don't actually understand exactly what it is doing to be honest).

[cvanem]

Summary

The components used with a Prisma configuration are similar to Graphcool, but consist of an additional seperate GraphQL server layer between the client and the Prisma GraphQL server layer. For our implementation (and what the boilerplate examples use), this GraphQL server layer is implemented with the graphql-yoga package and automatically bound to the Prisma layer with the prisma-binding package. This separation provides more flexibility in regards to middleware or custom functionality needed, such as third party authorization. For this specific project it means we will need to move some of the authentication and permission logic around. See here for more information. https://www.prisma.io/docs/maintain/graphcool-to-prisma/overview-gcf1/#new-architecture:-your-graphql-server-uses-prisma-as-a-database-layer

Advantages

• Newer technology.
• Built in database seed functionality.
• Added flexibility for GraphQL layer
• CLI tools appear to work better than the Graphcool CLI, at least on Windows.

Disadvantages

• Development migration cost
• Production migration cost

Hosting Options (Prisma Cloud)

• Free for up to 3 users
• Provides the interface for viewing and managing your database and deployment.
• The Heroku integration automatically creates a Postgres database and deploys the GraphQL and Prisma servers and hosts them on Heroku.

Required Tasks

1. Migrate project structure. Use one of the boilerplates for folder structure examples: https://github.com/graphql-boilerplates/node-graphql-server https://github.com/graphql-boilerplates/react-fullstack-graphql

2. Migrate data model. The data model modifications can be found here.

3. Populate seed file with some data, setup secret key and test deployment.

4. Migrate Resolvers. See Migrate Resolver Functions

   • Create resolvers for GraphQL server
   • Move authenticate functionality in graphcool.yml into a resolver
   • Integarate resolver with Auth0 code (server/src/auth0)

5. Migrate Permissions

   • Move permission logic from graphcool.yml to new GraphQL Server structure. See Migrate Permission Rules)

6. Update Documentation with new instructions

7. Migrate Production Database

Research

• [x] Determine what will need to be done to migrate the existing production database to the new deployment? We should be able to point the prism deployment to the existing production database. More research will need to be done to determine how this can be done. See Data Import and Export See Migrating to Prisma

• [x] Determine how Prisma will be used/deployed to production and what modifications will be necessary to support this. Prisma Cloud is free for up to 3 users. This provides the interface for viewing and managing your database and deployment. The Heroku integration allows you to automatically create a Postgres database and Prisma server that are hosted on Heroku with minimal effort. There are other hosting and deployment options available, which will typically require more configuration.

• [x] Determine how to create a development database with Prisma.
  The 'prisma init' and 'prisma deploy' commands can be used to setup the database. The commands are very similar to graphcool, except they actually seem to work on Windows 👍. Minimal modifications to the data model will be required to get this working. The data model modifications can be found here.

• [x] Determine how to seed the database.
  Prisma supports seeding by simply adding the seed file and a reference to the file in the prisma.yml file. The seed file is then populated with the desired seed data and the Prisma CLI can be used to perform seed operations as needed.

• [x] Determine how to migrate the permission structure.
  The permissions will need to be migrated. (See: Migrate Permission Rules)

• [x] Determine how login/signup will be handled. The authenticate function that is currently in graphcool.yml will need to be adapted or replaced for use with Prisma. (See: Graph Cool To Prisma: Resolver Section and Migrate Resolver Functions)

• [x] Determine, what if any modifications will need to be made to the client application to support using Prisma.
  I don't believe any other code will be modified. The existing logic and Apollo connections should work as is.

• [x] Determine if we need to use the Prisma secret key and if so, what changes need to be done to support this.
  Yes. The only change required would be adding the key to the new configuration. The secret key adds security to the Prisma layer which is good, especially when hosted on a public server.

[wdoug]

Awesome. Thanks for this research @cvanem! Did you by chance look into what database gets backs the prisma servers when you create an app with something like graphql create my-app --boilerplate node-basic? I'm curious what that is and if we could use one of those databases for prod or if they are only useful for development purposes.

[wdoug]

Also, I'm going to edit the title of this card to reflect the work that you did on the research side to make way for a followup spike of an actual implementation of this work.

[cvanem]

@wdoug I dug into that source code a little but but got stuck trying to trace down the --boilerplate argument.

My best guess to what it is actually doing is the same as running prisma deploy and auto-selecting the options for you and deploying to a public server. Let me investigate more and I will update this post.

[wdoug]

@cvanem it looks like you are right except that it finds whichever server is faster (see this line, called from here).

Do the Prisma docs say what the status of those public servers are?

[cvanem]

@wdoug I didn't see much on them. This has a bit on the demo servers, which I am guessing that is what is created with the boilerplates: https://www.prisma.io/docs/run-prisma-server/demo-servers-prisma-cloud-jfr3/

[dchao19]

@cvanem Thanks for the research. Based on what I've read, I think we'll still need some form of "application server" that consumes the Prisma Server (hosted on Prisma Cloud?) so we can handle authentication and authorization. Do you know if my assumption is right? I was thinking we could use a lambda function (sort of like we do now) if all it really does is handle authentication and then forward the GraphQL request to Prisma?

[wdoug]

I thought that prisma could also host that for you. For example, I'm pretty sure that in the graphql boilerplate it hosted both the prisma database setup and the graphql-yoga application server.

[cvanem]

@wdoug Yeah I think it is hosting them both. As far as I can tell when you use prisma deploy (with a prisma account) , it automatically links you to a heroku account and deploys them there and sets up the database.

[wdoug]

What if you don't have a prisma account? Will it prompt you to create one like they had for the graphcool framework?

[cvanem]

Yeah, it gives you some options to pick from. You can create a new account and authorize from the CLI. I believe there is also an option to just enter a server address in.

[cvanem]

Where is the production database hosted now?

[dchao19]

Ah nice. I must have gotten confused looking at this boilerplate and this example of using Now deployment to host the application layer. There's a lot of documentation out there, but sometimes it doesn't all say the same thing.

[dchao19]

@cvanem We use Graphcool and Graphcool Cloud. Graphcool cloud hosts the database and the GraphQL layer and the application layer we use for auth.

[cvanem]

@dchao19 Yeah the deployment section here specifically says you need to handle deployment of the GraphQL application layer...so I think I may have stated that incorrectly above in regards to the Prism Cloud deployment hosting both the application and Prisma layers.

I think I got confused as it is is working in my development environment, but in that scenario your PC would act as the application layer? I think I need to do a bit more research.

[cvanem]

@wdoug @dcha19 More detail on the Separation of Application and Database Layer The Heroku Integration I was referring to in Prisma Cloud only deploys the prisma layer.

So, if I understand, we would require a separate layer hosted somewhere in a production environment. graphql-yoga Deployment

The biolerplate examples require you to run yarn start on both the server (graphql-yoga) directory and the application directory. So in a development environment the local PC acts as the application layer and the client.

[wdoug]

Oh, I must have been confused because some of the boilerplates don't use the application layer at all and just have the prisma database layer...

Hmm. That's unfortunate. That means we would potentially have 4 different components for this: database, prisma server, application server, and frontend server right? We should probably look into our options for this before we move forward. I'd still like to see if we can take advantage of a free tier that doesn't sleep (i.e. not using heroku for the application layer).

I wonder if we could use Netlify Functions for the authentication/application layer since we are already using Netlify for the frontend. I suppose it might also be worth looking into moving auth to Netlify Identity instead of Auth0 since Auth0 has proved to not be as simple as advertised (although that would probably be a bigger headache - maybe we could do it later). Zeit now might also be a decent option for the application layer if that doesn't work. Then what? Would we use Prisma cloud for the prisma layer and database? Or prisma cloud for the prisma layer and something like the free tier on Heroku Postgres? Could we push the prisma data layer up to wherever we are hosting the application layer? It seems like it would be beneficial for those to be co-located, but maybe for our purposes, it wouldn't matter much?

Is this whole thing even worth it? Thoughts @cvanem, @dchao19?

[cvanem]

@wdoug The graphql-yoga examples indicate it is possible to use netlify-lambda for the application layer: https://github.com/prisma/graphql-yoga/tree/master/examples/lambda-netlify

It might be some work to get it working correctly...

As far as the Prisma layer goes, I believe you are looking at using docking containers if you want to host it somewhere else. Once you go this route, it gets harder to argue the benefits of migrating to Prisma.

Also, I don't think Prisma Demo accounts are intended to be used in a production environment. They will work but there are some restrictions. More on this here..

How has Graphcool worked out for you guys so far? If you haven't had any issues with it, I would be inclined to argue that the Prisma migration is not worth the extra headaches and added maintenance.

[wdoug]

It has worked decently. We did have a period previously where we were running into some shortcomings of it and were debating how to proceed (see #203). One of the options we were considering was moving to Prisma, and at that time we ultimately decided to just continue using the graphcool-framework and use the apollo-link-rest library to pull in external data from REST APIs on the client. We figured that since we could generally work around the limitations we were facing at the time it wasn't worth investing the effort in migrating it and dealing with the additional infrastructure when that effort could be focused on other features or other Code for Denver projects.

I am going to try to document some of the pros and cons that I see to migrating:

Benefits of moving to Prisma:

• Could do schema stitching with the github's graphql api and thus have a much better and more powerful interface to get data from github
• Could provide a graphql api to proxy to other data sources
• Generally a lot more control over the backend
• Would get built-in support for database seeding
• Much more easily debuggable and testable as well as faster and potentially simpler authentication workflows
• Potentially easier database management and backups
• Avoids the risk of the graphcool-framework being deprecated (previously I believe I read a comment that they were planning on keeping it going, but I can't help but notice the lack of much development and the fact that they just merged a series of docs on how to migrate to Prisma to the graphcool-framework repo)
• Better support for typescript
• Quite a bit bigger community
  • Example code for reference such as these prisma examples and these graphql boilerplates
• Other new features and increased performance and reliability

  New API features introduced in Prisma include batch mutations, improved nested mutations and transactional mutations and more.

Potential downsides

• The difficulty and amount of work required to migrate the backend
• Coordinating migrating the front-end see docs here)
• A more complicated infrastructure to maintain
• More complicated ecosystem and code (and the docs and examples aren't always up-to-date or clear on what is recommended)
  • Graphql server (probably implemented with apollo-server per discussions like these)
  • Resolvers (probably using some sort of schema delegation althought it's not exactly clear what the best approach is)
  • Authorization (after looking at a bunch of different options it seems like graphql-shield actually provides a fairly straightforward and relatively similar way to define access control - example with prisma)
  • Generally more code

Based on the above list, I'm on the fence, but think probably the appropriate product decision is to defer additional work on migrating to Prisma for now, while simultaneously knowing that there are several things that could prompt that decision to change in the future. Meanwhile, from a personal learning perspective I am fairly interested in working on a spike implementation of migrating to prisma although certainly not commiting to actually following through with it.

[wdoug]

I'm going to reopen this as we are still discussing it

[wdoug]

Prisma 2 will be easier to deploy and thus a little easier to migrate to in addition to getting the newer features