search

Js-Brecht / gatsby-plugin-ts-config

Configure Gatsby to use typescript configuration files
MIT License
60 stars 5 forks source link

Gatsby v3 #29

Closed openscript closed 3 years ago

openscript commented 3 years ago

Yesterday, Gatsby v3 got released. I've upgraded already two projects. They also use gatsby-plugin-ts-config. So far everything seems to work! Awesome 🥇

After some more testing I think the peer dependency should be updated and a new version released.

warn Plugin gatsby-plugin-ts-config is not compatible with your gatsby version 3.0.0 - It requires gatsby@~2.x.x
Js-Brecht commented 3 years ago

I'm glad to hear it works with v3. I think this deserves further investigation before the peer dep is updated, though. For example, I want to make sure some of features are not broken by the proxy model this plugin implements, like incremental builds.

openscript commented 3 years ago

I'm happy to help, but I think I need a little help from you. What exactly you want to check and where should I start? :)

Js-Brecht commented 3 years ago

Mainly I just wonder if Gatsby will pick up changes in gatsby-browser && gatsby-ssr if they are hosted through this plugin. It doesn't follow a standard model, and it appears at first glance that Gatsby is doing static analysis of the files to see if they've changed, and using that information to determine if pages should be rebuilt.

In the case of gatsby-browser/ssr, changing them will probably force your entire website to rebuild. However, if Gatsby isn't finding them in the place it expects them, then there's a good chance that it won't pick up their changes and thus rebuild the site. This could result in some weird bugs where pages aren't rebuilding, or changes you're expecting to show up at the root of your pages aren't... and it may be difficult to track down why without warnings.

This plugin desperately needs a v2, which is going to entail a complete redesign of its general architecture... and I have been thinking for some time about the best way to do this redesign.

It started as a small POC, and that initial model causes some incompatibilities with Gatsby itself. For example, that "proxy" method I mentioned... when Gatsby calls this plugin's gatsby-config, it reads in (and transpiles in memory) your gatsby-* files, then returns those values it transpiled as its own; this is why I call it a proxy... It does the same thing with your gatsby-node, and it uses its own gatsby-browser/ssr files along with some webpack aliases to re-export yours. One challenge of this design was sharing site/plugin configurations & options across disparate parts of the Gatsby lifecycle, which I solved by using singletons. Since it is using singletons, it does not support themes. Also, because Gatsby doesn't actually know about your files, it can't look at them to watch for changes.

Another compatibility issue is plugin deduplication... Gatsby relies on the resolve + options properties of plugins in its fully compiled plugin config to deduplicate. This plugin currently takes your entire gatsby-config and changes the resolve property to a fully-qualified path in order to make it compatible with yarn2/pnpm. This is an already existing problem, so not directly related to Gatsby v3, unless they've implemented new plugin overrides like they do for gastby-plugin-page-creator and gatsby-plugin-typescript... However, it could still wind up causing lot of unforeseen issues, depending on how much they've changed plugin processing in v3. I do have a PR open to fix this, but it exposes new public api for plugin configs, so I'm not sure if they're going to accept it.

How I picture this changing is moving towards a hybrid static/dynamic transpiling process.

I was hoping that by v3 they would have implemented Typescript support for gatsby-* config files into the core, but... still no, it looks like :man_shrugging:. The alternative to a complete rewrite of this plugin is integration into the core, which would be a major process that I just haven't had the time to tackle yet.

openscript commented 3 years ago

Thank you Jeremy (@Js-Brecht) for outlining the situation so precisely.

I've came along some issues you already predicted. When developing a website and using gatsby-plugin-ts-config the page won't reload sometimes or is displaying that it's executing queries in the background, but it never comes to a result.

Personally I would vote for an integration into the Gatsby core or the gatsby-plugin-typescript itself. Some time ago @KyleAMathews wrote (https://github.com/gatsbyjs/gatsby/issues/1457#issuecomment-352585835) that Gatsby will probably never have support for that. In my opinion it's a necessary feature to have full fledged TypeScript support also within the config files. Using other measures to checking code like with PropTypes and stuff, isn't a real solution.

Js-Brecht commented 3 years ago

I agree it is important, considering how extensive these config files can become.

The thing about gatsby-plugin-typescript is that it’s still just a plugin, so it will fall prey to similar issues. It’s just one that’s maintained by the core team

dvakatsiienko commented 3 years ago

Agree that the functionality would be great to see in gatsby core. Otherwise to have support for Gatsby 3.

openscript commented 3 years ago

@Js-Brecht Do you have any contact to the Gatsby team? Probably it would be a good idea do have a clear, well-thought and as short as possible new issue in the Gatsby repo.

By the way, I've now two projects in production which use gatsby-plugin-ts-config and are on Gatsby 3.1. No issues at all so far!

dvakatsiienko commented 3 years ago

@openscript Voting up for your initiative! :)

Great to hear that you managed to get this plugin to work. You you share an instruction how to achieve that?

Currently the plugin says:

warning Plugin gatsby-plugin-ts-config is not compatible with your gatsby version 3.1.1 - It requires gatsby@~2.x.x

and erroring out afterwards.

openscript commented 3 years ago

Well I'm having the same warning, but everything works. Do you have error messages?

dvakatsiienko commented 3 years ago

Yes, but it is hard to explain — as for now getting in touch with getsby. I'll add here when I'll get something to work.

Js-Brecht commented 3 years ago

@dvakatsiienko The peer dependency warning shouldn't hurt anything. There could be other things going on, though, so would need to see what it's saying

@openscript If you're referring to moving this into the core, it was mentioned by one of their team on a Typescript integration topic, but I might need to do an RFC because to do it myself I would want to provide control over how the transpiling is done. I usually use ts-node because of the added functionality it gives me with ts transformers, while I'm certain the Gatsby team will want it done with Babel only. To have this level of control over what transpiler to use, and the fact that it needs to be figured out before gatsby-config is processed, means figuring out a new way to configure Gatsby.

Any time there's new public api going in, it's a process. I'm still waiting on a PR to get either rejected/merged that would let me resolve the issues I mentioned above (it's been months).

openscript commented 3 years ago

@Js-Brecht If I can support you somehow, let me know. Do you have a link to the PR you mentioned?

cameronbraid commented 3 years ago

I am also getting this error 

 ERROR #11329 

Your plugins must export known APIs from their gatsby-node.js.

See https://www.gatsbyjs.org/docs/node-apis/ for the list of Gatsby node APIs.

- The plugin gatsby-plugin-ts-config@1.1.5 is using the API "createSchema" which is not a known API.

Some of the following may help fix the error(s):

- Rename "createSchema" -> "createSchemaCustomization"
Js-Brecht commented 3 years ago

@cameronbraid this plugin takes your gatsby-node, transpiled it and sends it back to Gatsby. The reason it says that it’s this plugin that is using an invalid api is because of the proxy method that is mentioned earlier in this thread.

It passes all of the exports back, so I would suggest examining your gatsby-node and make sure you aren’t exporting anything from it that isn’t a standard Gatsby api.

cameronbraid commented 3 years ago

Ah, yeah - my bad ! Thanks for pointing that out.

All good now :)

Js-Brecht commented 3 years ago

@openscript I will for sure if i get unblocked, thanks for offering. The PR in question is here: https://github.com/gatsbyjs/gatsby/pull/28523

I think if that gets merged, I’ll be unblocked for changing the process to dropping transpiled files into the cache. This plugin will just return a gatsby-config that points to each of them, which means gatsby can also watch them for changes... that’s the best choice.

Another choice I’ve been considering is doing a couple of function endpoints that will be used inside of your gatsby-config.js (the generateConfig function, essentially) and your gatsby-node.js. Each of these would just point to their respective .ts file, transpile it and return the value as is. It would make things simpler and it would support themes, but it would still lose out on file watching/static analysis and there’d need to be some work done to make sure the state between gatsby-node and where it starts in gatsby-config stays in sync. It also means that you still wind up with all the gatsby-* files in your root, which I’m not a fan of.

KyleAMathews commented 3 years ago

👋 since I was mentioned I'll pop in 😆

My comment there was written 2.5 years ago pre use of typescript for anything in gatsby. We're a small focused team so I doubt the core team will get to it immediately but I'd love a community PR for first class support for typescript for gatsby's special files.

Js-Brecht commented 3 years ago

Glad to hear from you, @KyleAMathews! Since you're here, would you mind sharing your opinion on having (opt-in) support for ts-node (or a ts compiler, if JIT transpiling isn't desirable)?

That's been one of the things stopping me from putting any work into integrating this in the core; there was a thread in the issue logs somewhere we were discussing it and using ts at all for this seemed to be generally opposed. I really like the extra functionality (transformers) + type checking I get with an actual ts compiler... stuff that you just can't do with babel. I understand the performance considerations, but the extra 4 seconds it takes to spin up a Gatsby build is, to me, worth the added functionality.

Everspace commented 3 years ago

I love using typescript to do things with say, joi or zod to catch myself doing naughty things in frontmatter or yaml that builds into the eventual site, and make sure I produce the correct types in the api land before we start the site proper. One of the big gotchas I have felt for gatsby is just... yea the front-end is great, but doing anything yourself for data assemblage is misery.

What prior art is there for config available in ts and js? Is it babel or ts-node?

Js-Brecht commented 3 years ago

What prior art is there for config available in ts and js? Is it babel or ts-node?

@Everspace I'm not sure I understand the question. This plugin allows either babel or ts-node. I've also started v2 that should close this ticket and a couple others

cometkim commented 3 years ago

I used to use babel register in v2.

https://github.com/devsisters/gatsby-starter-typescript-workspace/blob/4f2a4f0/site/gatsby-config.js

Since v3 it doesn't work, so I had to use cli to transpile gatsby/gastsby-* files first.

And Gatsby v3 is completely broken in P'n'P so I can't migrate my project. I'm surprised that Gatsby has long been a leader in ensuring P'n'P compatibility. I can recover local builds with P'n'P by some dirty workarounds, but Gatsby Cloud uses a separately built Global CLI, so it's irreplaceable.

I am curious about the future roadmap.

cometkim commented 3 years ago

Sorry, I was mistaken for the Gatsby roadmap discussion here.

Js-Brecht commented 3 years ago

@cometkim about the api types, check out https://github.com/gatsbyjs/gatsby/pull/30819

Js-Brecht commented 3 years ago

To everybody watching/participating in this issue, I have begun work on v2. It should close this issue, as well as several others. Please see #31 for more details