GraphQL

[berekuk]

Pushing this just to show the progress so far. Not ready to merge yet.

Related issues: #32, #59

Notes:

1. Pothos is good! Some minor issues with docs, but nothing critical; Prisma plugin and Relay plugin work fine, which is great.
2. urql with SSR is... ugh. Dealing with it right now (haven't pushed the related commit yet). I had to read through the next-urql source code to figure it out.
   • next-urql uses the deprecated getInitialProps from older Next.js as a default mechanism but has the solution for more modern getStaticProps or getServerSideProps
   • we'll have to list all requests for GraphQL data in getServerSideProps manually (e.g., deeply nested components which need some portion of data won't magically SSR with all their useQuery data by themselves)
     • this is not urql's fault, since the alternative is to render everything on the server twice, which is slow
   • I don't like the amount of code which will be necessary for every getServerSideProps function (urqlState: ssrCache.extractData(), initUrqlClient, etc.); I'll try to come up with some helper functions to make the DX smoother
3. I'm replacing searchAccordingToQueryData with a GraphQL query, to abstract away all search logic from the frontend code. This requires substantial refactorings, hope to push the working implementation in a day or two.

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
metaforecast	✅ Ready (Inspect)	Visit Preview	Apr 21, 2022 at 8:30PM (UTC)

[netlify[bot]]

❌ Deploy Preview for metaforecast failed.

Name	Link
🔨 Latest commit	3ef786db4bdd1e456c6c890e2fe766ff7752efe6
🔍 Latest deploy log	https://app.netlify.com/sites/metaforecast/deploys/6261be86cb55ef00089a48e4

[berekuk]

80% complete, 80% to go.

• SSR is working
• frontpage uses graphql, dashboards too

Needs fixing:

• [x] timestamp field (I used custom scalar for the timestamp field on the backend and need to map it properly on the frontend now, probably with urql-custom-scalars-exchange, but it requires schema awareness which I haven't dealt with yet)
• [x] creating dashboards is not implemented yet
• [x] there are probably a few bugs, I should check every page by hand
• [x] docs for writing new endpoints and description fields in graphql schema
• [x] fix Vercel deployments (need to run prisma generate somehow; https://www.prisma.io/docs/guides/deployment/deployment-guides/deploying-to-vercel)

Also:

• I don't like my current implementation for visualization and qualityIndicators, not sure if I'll improve it before merging or deal with it later
• I feel like CommonDisplay can/should be simplified, though it's not much worse now than it was before

[berekuk]

urql-custom-scalars-exchange

It works, but it's incompatible with SSR, since Next.js can't serialize Date objects.

There's a workaround with superjson, but I don't think that an extra dependency is worth it.

I left urql-custom-scalars-exchange config & schema introspection code for the future since I already wrote it when I realized this, and schema introspection might be needed for better urql caching anyways.

For now I'm just serializing timestamp as an int (which is better/safer than a date string) and decode it with new Date(ts * 1000) on the frontend.

creating dashboards

Done.

---

Pushed the new changes, https://metaforecast-git-graphql-quantified-uncertainty.vercel.app/ is mostly functional now. (SSR on the frontpage is currently broken, will investigate).

[berekuk]

Ok, I think this is done!

Docs are in docs/graphql.md, and I added a description for the most of graphql fields (sometimes copying from specification.json, which can be deprecated soon, since GraphQL is better for this kind of documentation).

---

Notes on SSR.

I still don't like how SSR is implemented, especially making a choice of "do I pass the page data from getServerSideProps in urqlState cache or in page props".

Passing in props means that the page is not really dynamic (I consider initialBlah props with useState to be an anti-pattern which should be avoided if possible). Which is mostly fine, except for the frontpage, which is dynamic. It also means that we won't be able to benefit from urql's caching abilities in the future, and caching is a big selling point for graphql; though it's more useful for SPA-like rich UI features, which we don't have yet.

On the other hand, passing in urqlState means that we have to do the query in getServerSideProps and then do the same query with the useQuery on client. Which means that if the document or variables for one of these queries ever changes on the future and we forget to change the other call then SSR would break (i.e. degrade to client-side query, the page won't break entirely, but it would break silently and we might not notice it for a while).

Issues with both of these approaches are non-critical, and it's not hard to change the code from using one approach to another. But I don't like having this choice, it feels like extra cognitive load for no clear benefit.

I don't have any good ideas for how to deal with it yet, and maybe I'll just get adjusted to it over time.

---

Notes on types.

• I got rid of FrontendQuestion since we can just use types from graphql-code-generator based on fragments in our queries
• OTOH, there are now two Question types on the backend: first in src/backend/platforms/index.ts and the second one from Prisma; I'd like to keep just the second one eventually, but we'll need to refactor the DB to make it strictly typed (split qualityIndicators and extra fields into postgres columns)
  • for now, dealing with JsonValue for qualityIndicators and extra is annoying, since Prisma assumes that a JSON field from the DB can be any JSON-compatible value, e.g. a scalar or an array
• I got bitten a few times by "strict": true not being enabled in our tsconfig.json, especially by nullable types; we're down to 349 errors when strict is enabled (from 523 on the initial typescript PR)

[NunoSempere]

Any idea why the netlify build preview is failing? The vercel one is working: https://metaforecast-lfn840cvn-quantified-uncertainty.vercel.app

[NunoSempere]

Ok, I think this is done!

Should I merge, or any last checks?

[NunoSempere]

Your notes on SSR (server-side rendering, I'm guessing) went a bit over my head. Any actionables, besides writing this down somewhere so that we remember this when we come back to it in a few months?

[berekuk]

Any idea why the netlify build preview is failing?

Probably because the build now requires prisma generate, which is run in npm run build (see scripts in package.json) but netlify runs next build instead.

Anyway, I disabled netlify builds yesterday since we don't use it anymore.

Your notes on SSR (server-side rendering, I'm guessing) went a bit over my head.

That's fine. It's kinda hard to explain clearly without repeating all graphql/urql docs, and I think I'm not doing anything non-standard here.

Any actionables, besides writing this down somewhere so that we remember this when we come back to it in a few months?

No, my notes above are "writing this down somewhere", I should've been more clear about that. No actions needed. (I slightly dislike putting stuff like this in docs/ in the repo, since code usually changes faster than documentation and documentation can get out of sync with reality or with our understanding of best practices.)