Gatsby Explanatory Diagram

[SachaG]

For the new homepage, I think it'd be nice to have some kind of diagram to explain the whole pipeline, and the fact you can mix and match different data sources. Here's a start:

https://www.figma.com/file/J6IYJEtdRmwJQOrcZ2DfvxDD/Gatsby?node-id=1%3A23

[KyleAMathews]

I like it! That'd be perfect.

I'd suggest moving GraphQL from "get data" to "build". Perhaps GraphQL could be another box where all the data option lines go into then into Gatsby. And instead of API, put "CMS" as headless CMSs will be a popular pairing with Gatsby.

[SachaG]

It doesn't feel very intuitive to me to have my markdown or YAML data go through GraphQL though. Isn't it more straightforward to introduce GraphQL as another potential data source at the same level as the others?

[KyleAMathews]

No, GraphQL actually makes a bad data source for sourcing data into Gatsby because GraphQL doesn't have a way to say "give me all data" like REST APIs do.

But Markdown & YAML do go through GraphQL. The flow of data in v1 is source plugins grab data from different places (APIs, file system, databases, etc), then transformer plugins transform the data in various ways (e.g. markdown to html), then react components have GraphQL queries which pull data out of the data layer into the component.

So regardless of the data source, all data (if it's used in the site) goes through GraphQL.

Or another analogy. The data layer is a way to build a database on the fly out of your data and GraphQL is the query language for querying data out of that database.

[SachaG]

So does this means you can't just import a YAML file like you could in the previous version? I thought this was actually a really cool feature, having to put that YAML file through GraphQL seems like an unnecessary step to me?

[KyleAMathews]

You totally can. Just now you have options. Putting the yaml file in the data layer means you can query just parts of the data, you can sort, filter, transform the data. And you can link the data to other data e.g. can create thumbnails of authors with https://github.com/gatsbyjs/gatsby/blob/1.0/docs/blog/author.yaml in graphql e.g. in https://github.com/gatsbyjs/gatsby/blob/1.0/www/src/pages/blog/index.js (see query at bottom)

On Wed, Jun 14, 2017, 7:49 PM Sacha Greif notifications@github.com wrote:

So does this means you can't just import a YAML file like you could in the previous version? I thought this was actually a really cool feature, having to put that YAML file through GraphQL seems like an unnecessary step to me?

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/gatsbyjs/gatsby/issues/1172#issuecomment-308613690, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEVh7mVCV_8212Ebzy8CHypb72CWpi9ks5sEJvNgaJpZM4N6nAH .

[SachaG]

I think having options is great, but I just think we should avoid the perception that you have to know GraphQL to use Gatsby, because that might put off a lot of people.

[KyleAMathews]

avoid the perception that you have to know GraphQL to use Gatsby, because that might put off a lot of people.

Ooo good point. Yeah, it's actually not and I agree, no reason for people to perceive their being barriers where there's not.

I think Gatsby is pretty analogous in a lot of ways to the old php/mysql days. Many people could and did avoid ever directly writing SQL queries because frameworks sprang up which abstracted away most of that.

The same is true of Gatsby that underneath there's queries to get your data but it's very possible to abstract a lot of that away.

Right now things are a bit raw but especially once theme support is added, many people will be able to build fairly sophisticated sites without touching graphql.

That being said, learning GraphQL is highly supported as it's not that hard to learn and there's a lot of powerful things you can do with it but yeah, it being there shouldn't stop anyone from playing around and building things with Gatsby.

One of my first TODO items after the beta launches is to build an abstraction similar to how <v1 Gatsby works with wrapper components. If you add a markdown file to src/pages it'll get turned into a page automatically and the data for it will be injected into a md.js component in a wrappers directory.

[0x80]

I have doubts about the simplicity of the diagram, but it might help as an introduction for total newcomers to this JAM stack approach. I agree that we should probably keep graphql for another diagram to avoid scaring people. The column on the right is all the same "static hosting" to me, but I also don't know what else to put there.

A more technical diagram about the architecture of Gatsby including graphql and the plugin system would be really helpful I think. It should also illustrate that the served static files boot into an SPA, and the notion of a PWA.

[SachaG]

I have doubts about the simplicity of the diagram

Do you mean it's too simple, or not simple enough? My idea was to quickly lay out what Gatsby can do, especially the fact that it can consume multiple data sources and is very easy to deploy compared to WordPress, Drupal, etc.

[0x80]

@SachaG I mean maybe it's too simple. Because it doesn't show anything special to Gatsby. It basically says "you can transform markup files with React to static html and serve that". There are other tools that do very much just that.

[0x80]

Actually that's not really true. You mention json and graphql already. But yeah my point was that it's a fairly simple diagram, and it doesn't tell you much about what makes gatsby special. But maybe as a first introduction that is fine too.

[SachaG]

Yeah I agree. I just think we have to explain Gatsby in a way that makes sense to the widest audience possible. If we want to highlight its specificities we could maybe have a separate "Gatsby vs…" or "what makes Gatsby special" section?

[djm]

I love this idea. It is simple yes, but I think it offers a good top level overview of what Gatsby does rather than the intricacies of how it does it. Where React takes over, the plugins, the service worker, and the offline-first abilities can definitely be in another more technical diagram that isn't the first thing that is shown to users. Having said that (😁)...

I can see why leaving GraphQL out of this diagram may be a good idea; but for me, it was what attracted me to Gatsby in the first place - the very idea that you can use a unified interface to query a variety of different data sources is huge, and as far as I'm aware - no other static generator can just as easily pull from a headless CMS at the same time as pulling from on-disk files. The querying capabilities of GraphQL give unparalleled control over exactly what bits of your data sources are displayed where. Having used many other generators, this is not a common thing so I think we should shout from the rooftops about it - as it means you can build pretty much any content based site rather than just boring blogs.

I took the liberty of playing around with figma & mixing in some suggestions from this thread, and ended up with this:

on figma.

[erutan]

The data sources aren't mutually exclusive per build, but the deploy (realistically) would be. It seems weird to have multiple sources exiting the build process. I'd have the target be static hosting, then have gh-pages and netlify be examples.

Keeping to the current visual styling, here's a quick edit (though I doubt it's the best way to show those as examples).

[erutan]

This is a bit clearer, though not the prettiest treatment.

Maybe change the label for APIs from CMS to Services?

https://www.figma.com/file/tSJTPj6GXJXcmBbhfJGvx3/Gatsby-deploy-tweaks (as an aside, it'd be cool if figma had a fork history chart ala github)

[KyleAMathews]

@erutan agree this makes more sense. Played with it a bit myself :-)

https://www.figma.com/file/EbS9ndNreKTAeCbutdyokH/Gatsby-deploy-tweaks-Copy

[erutan]

It's more balanced and comprehensive - one minor quibble is that nginx is a server not a host (unless I've missed something). 'Self-hosted' perhaps?

[KyleAMathews]

Yeah self-hosted works better.

[sebastienfi]

Maybe under CMS APIs add the logos of existing CMS source plugins ?

Wouldn't it be nice to include the provider model into this schema ? This could display the most useful plugins, like lazy-loading, etc.

[KyleAMathews]

People commenting on this issue — @fk has been reworking the diagram to use html/css in #1605 — see the latest preview build https://deploy-preview-1605--gatsbyjs.netlify.com/

Would love to hear your thoughts over on the PR!

[sebastienfi]

@KyleAMathews That looks awesome ! Tested on Chrome Windows, IE, iOS, and Android OK !

[KyleAMathews]

The diagram has been great! Thanks everyone who helped!