search

vitejs / vite

Next generation frontend tooling. It's fast!
http://vite.dev
MIT License
68.55k stars 6.19k forks source link

Replace `$ vite build && vite build --ssr` with `$ vite build` + enable plugins to add build step #5936

Closed brillout closed 2 years ago

brillout commented 2 years ago

Clear and concise description of the problem

We want to build deploy plugins, e.g. vite-plugin-cloudflare-workers, vite-plugin-vercel, and vite-plugin-deno-deploy.

Currently, the user would need to:

// package.json
{
  "scripts": {
     "build": "vite build && vite build --ssr && vite-plugin-cloudflare-workers build"
  }
}

Suggested solution

// package.json
{
  "scripts": {
     // Automatically runs all three build steps:
     // 1. client bundling
     // 2. SSR bundling
     // 3. custom build step (defined by the plugin)
     "build": "vite build",
  }
}
// vite.config.js
import cloudflareWorkers from 'vite-plugin-cloudflare-workers'
export default {
  plugins: [ cloudflareWorkers() ],
  server: './path/to/server.js'
}

Additional context

There is an increasingly number of deploy environments.

This ticket enables deploy plugins which in turn enables deeper collaboration between SSR frameworks.

Eventually it is the deploy providers who will maintain these plugins. (Like how Cloudflare Workers took over miniflare.)

Validations

Aslemammad commented 2 years ago

I'm going to put some time on this one. See if I can tackle it down! Thanks.

brillout commented 2 years ago

@aleclarson Thoughts on this? Could be relevant for https://github.com/alloc/viteflare.

patak-dev commented 2 years ago

@brillout would you extend the API you are proposing? In your config example, server is used but that is the Vite Dev Server config already. And how do the plugins communicate that they have a build step that should be executed?

brillout commented 2 years ago

I'm glad this is getting attention. There is a big need for shared deploy adapters & plugins that the whole Vite ecosystem can use. Instead of having vps, SvelteKit and Astro re-invent the wheel. (@cyco130 is experimenting on something quite exciting in that space, and @magne4000 is working on vite-plugin-vercel which supports Vercel's ISR platform.)

I think the writeBundle() hack is, actually, quite neat. It's flexible, as you can see.

Being able to append tasks to $ vite build is AFAICT enough. So I'm proposing we introduce a new Vite hook onBuildEnd() that enables the writeBundle() hack in an official way: onBuildEnd() is guaranteed to run last and supports side effects. That way plugins and frameworks can safely append tasks to $ vite build.

The neat thing here is that we can re-use Vite's enforce mechanism to resolve plugin conflicts. For example, vps would use enforce: pre while the deploy plugins use enfore: post, so that the sequence is: 1. client bundling, 2. server bundling, 3. pre-rendering, 4. deploy specific build (e.g. ncc for Next.js or Cloudflare Workers' wrangler for bundling the server-side into a single worker code).

I'd suggest to experiment with the writeBundle() hack for a while and see how it goes. If it works well let's go for an official support with something like onBuildEnd().

As for defining new commands, I don't see much complications here. One thing though: other than $ vite deploy, I don't see a use case for defining new commands. So I'm thinking we could simply add a new $ vite deploy command that does nothing by default; a plugin has to define what it does.

benmccann commented 2 years ago

vps

Just FYI, this acronym is a bit confusing / overloaded because we also use it for vite-plugin-svelte

There is a big need for shared deploy adapters & plugins that the whole Vite ecosystem can use. Instead of having vps, SvelteKit and Astro re-invent the wheel.

It's gotten a bit more complicated on our side. SvelteKit has introduced server-side route splitting. This means that some of SvelteKit's adapters now have to know how to map SvelteKit routes to the routing functionality of each hosting platform. As a result, it'd be hard to share given the need for SvelteKit-specific logic. Perhaps there could be general adapters and each framework could provide their own route-splitting functionality, but it's gotten more complicated at least. Still, there are adapters that don't do route splitting like adapter-node that could likely be shared more easily.

aleclarson commented 2 years ago

This means that SvelteKit's adapters now have to know how to map SvelteKit routes to the routing functionality of each hosting platform.

I've talked a bit about Saus recently (for those who missed it: link), which could provide routing primitives and route-based code splitting if Vite decides not to go that route (no pun intended). The vision is that full-stack frameworks share a lot more in common than they're currently taking advantage of, so there should be a layer like Saus that helps with that. Such a layer would be a lot better of a candidate for a deployment adapter API that every Vite-based framework could use (indirectly through plugins). Finally, there was a Twitter thread all about this direction just recently (link).

cyco130 commented 2 years ago

Here's the work I've done that @brillout refers to: https://github.com/cyco130/vavite (in particular the @vavite/multibuild and @vavite/multibuild-cli packages, in the last paragraph of the readme).

patak-dev commented 2 years ago

I agree that given the current parallel explorations happening it may be too soon to formalize things in Vite core. @brillout your idea of a new onBuildEnd hook is interesting, but it feels a bit hacky to me. Shouldn't the non-ssr build and the ssr one be more on the same level? How do we offer the user the possibility to parallelize the build steps. And @cyco130's multibuild buildSteps may have the same issue, no? I see the appeal to have this in core though, as everyone in the ecosystem will end up reinventing the wheel if not. But looks like we are getting into the business of a task runner, and we should then have a task DAG. Maybe instead of a onBuildEnd we need a registerBuildTask hook if we take this responsibility in Core.

patak-dev commented 2 years ago

As for defining new commands, I don't see much complications here. One thing though: other than $ vite deploy, I don't see a use case for defining new commands. So I'm thinking we could simply add a new $ vite deploy command that does nothing by default; a plugin has to define what it does.

Is the idea that this command would execute a deploy hook from plugins? Wouldn't we have the same issue of needing a DAG to control what can be done in parallel?

cyco130 commented 2 years ago

And @cyco130's multibuild buildSteps may have the same issue, no?

My idea was to extend it later with requires or something like that to support parallelization (effectively turning it into a DAG task runner like you describe).

brillout commented 2 years ago

Do we really need a DAG? I do agree that a real task runner needs to be a DAG, but we only have a handful of tasks.

Concretely, $ vite build will always be composed of max 3 build steps:

Also note that pre-rendering needs to await Rollup bundling; we cannot parallelize these two.

For only three steps, I don't think we need a DAG. But maybe there are more steps I'm not foreseeing.

I propose we start without a DAG at first and see how it goes.

Shouldn't the non-ssr build and the ssr one be more on the same level?

Good point. I'm 👍 for replacing $ vite build && vite build --ssr with $ vite build + a new config. Although, I think it's lower priority and things are working out so far. I suggest to consider deploy adapters & plugins to be the highest priority thing here. We can always parallelize and "DAGify" things later.

The world is ripe for deploy adapters. If Vite delivers on this, that's big. Guillermo Rauch (Vercel CEO) is supportive, and Cloudflare Workers, I'm confident, will shortly follow, and with it all other deploy providers.

a new onBuildEnd hook is interesting, but it feels a bit hacky to me.

That's how I felt at first as well, but I ended up liking it. Because it fits well with the current Rollup plugin architecture. it's simple and it works. We can use the enforce mechanism instead of a DAG. Yes it's cheap but, for only 3 build steps, it works.

Anyways, in the meantime, let's see how things go with the writeBundle() hack.

Is the idea that this command would execute a deploy hook from plugins?

Yes.

Wouldn't we have the same issue of needing a DAG to control what can be done in parallel?

Only one plugin is allowed to define the deploy() hook. For a given app, there would be only one deploy plugin. E.g. either vite-plugin-vecel or vite-plugin-cloudflare but not both. There may be some edge cases that warrant the use of two deploy plugins, but I think it's ok for now to not support these edge cases.

Also CC'ing @Aslemammad who's working on vite-plugin-cloudflare.

brillout commented 2 years ago

As for the off-topic discussions, see:

benmccann commented 2 years ago

SvelteKit has been updated so that all building occurs within a Vite plugin. Calling vite build && vite build --ssr seems too clunky so we're instead programatically invoking vite a second time from the writeBundle hook

brillout commented 2 years ago

SvelteKit has been updated so that all building occurs within a Vite plugin. Calling vite build && vite build --ssr seems too clunky so we're instead programatically invoking vite a second time from the writeBundle hook

Same for vite-plugin-ssr 0.4.

I believe Rakkas also does this (although it uses another hook IIRC).

Let's see how this pans out.

brillout commented 2 years ago

I just released vite-plugin-ssr 0.4 that includes the writeBundle chaining trick.

Implementation: https://github.com/brillout/vite-plugin-ssr/blob/514843f92af77324d5038169ea2f45bfe7db7816/vite-plugin-ssr/node/plugin/plugins/autoFullBuild.ts.

Let's see how it works out.

benmccann commented 2 years ago

SvelteKit has five build steps that must be run sequentially as each depends on the output of the step before it.

One way to support this would be if Vite supported an array of configs just as Rollup does. Our builds could potentially look something like:

export default [
  {
    plugins: [svelteKitClient()]
  }
  {
    plugins: [svelteKitServer(), svelteKitPrerender()]
  }
  {
    plugins: [serviceWorker()]
  }
  {
    plugins: [adapterVercel()]
  }
];

An alternative solution may be to have Rollup change writeBundle or closeBundle from parallel to sequential for Rollup 3 (semi-related issue https://github.com/rollup/rollup/issues/2826) or to have Vite add a new sequential hook somewhere towards or at the end of the build (one proposed PR for that here: https://github.com/vitejs/vite/pull/9326). Then we could keep doing what we're doing while running into fewer ordering problems between the plugins. There's also a few other solutions that Dominik listed though only the second one is not extremely hacky: https://github.com/vitejs/vite/pull/9326#issuecomment-1193282124

Right now, we're kicking off the server, prerender, and service worker from writeBundle and adapter from closeBundle. The client, server, and service worker each require bundling. Some of the adapters are running a bundling step as well with esbuild (depends on the target platform and user options). The prerender step is just writing files out and doesn't do bundling.

We've reached the limits of this approach. We're trying to let people optionally switch out our service worker implementation for vite-plugin-pwa, but there's no way to make this work. The service worker generation has to happen after prerendering is run and before our deploy adapters run. However, writeBundle and closeBundle are both parallel hooks, so there's no way to enforce this. If vite-plugin-pwa uses writeBundle there's no guarantee our prerendering has finished. If vite-plugin-pwa uses closeBundle there's no guarantee it will finish before our adapter begins.

Allowing these to be separate builds or have more build steps would also allow us to refactor more of this to occur in shareable Vite plugins. E.g. as @brillout mentioned above, if adapters were Vite plugins then they could be shared across the ecosystem.

I've seen others running multiple bundlings as well. E.g. in addition to the folks that have already been mentioned in this thread above, vite-plugin-pwa uses a hook to kick off a new bundling.

Trade-offs between multiple builds in config vs chaining builds with a hook:

aleclarson commented 2 years ago

@benmccann Is there actually a need for multiple builds in vite.config.ts files, or is support through programmatic API enough? That would avoid the breakage of other tools that load Vite configs.

benmccann commented 2 years ago

@aleclarson we need to run multiple builds, but they don't have to be defined via vite.config.ts. Other solutions like the addition of a sequential hook would work as well or possibly even making enforce: 'pre' and enforce:'post' group and await plugins when executing parallel hooks (https://github.com/vitejs/vite/pull/9326#issuecomment-1193282124). Open to other ideas as well if you had something else in mind when referring to programmatic API. I think I'm probably leaning towards a new hook as being my favorite solution

antfu commented 2 years ago

writeBundle closeBundle parallels executing

I agree this is something limiting the ecosystem to grow and innovate. And we should figure out a better way to improve it. While changing it to complete sequential will hurt the performance and probably be a significant breaking change to the ecosystem. I'm thinking maybe we could have it per-plugin level control of whether those plugins should be executed parallel or sequential. We might need to figure out a good design of how we could express it by extending the current API.

Multiple builds in vite build

In general, I am against doing this. Even if it can be possible, I would personally think it's a bit anti-pattern. Vite as a front-end tool is mainly for building the client app, given its pretty low-level, meta frameworks calls build() from vite programmatically would assume it only performs one build. Frameworks like VitePress or Vitest will reuse the vite.config.js. Allowing a plugin to change its behavior of it is a bit risky for me.

I think multiple builds should be handled by upper-level frameworks to call Vite programmatically (multiple times at different timing), just like how we do it in many frameworks now. If we really want to rescue the CLI from Vite, I guess we could introduce a separate command like vite pack, but that would be a different story.

haoqunjiang commented 2 years ago

I'm not sure if vite pack is a good idea. I'm afraid most users would confuse it with vite build, then we still need a mechanism to warn them to use the correct command. If that's the case, why not just tell them to use another CLI tool?

brillout commented 2 years ago

RFC by antfu: https://hackmd.io/WgkOsRmpT0e5ACJHc0Dh6Q

antfu commented 2 years ago

RFC here: https://github.com/vitejs/vite/discussions/9442 (solving the limitation of writeBundle closeBundle in general). I am still leaning to against multiple builds in vite build.

brillout commented 2 years ago

RFC summary: https://github.com/vitejs/vite/discussions/9442#discussioncomment-3303445.

New discussion for #9496 - Should $ vite build also build SSR?.

antfu commented 2 years ago

Closed as https://github.com/vitejs/vite/discussions/9496#discussioncomment-3384709