[BUG] Snowpack builds before scripts run

[amithm7]

Bug Report Quick Checklist

• [x] I am on the latest version of Snowpack & all plugins.
• [x] I use package manager npm
• [x] I run Snowpack on OS Linux
• [x] I run Snowpack on Node.js v12+

Describe the bug

On snowpack 3.3.6 (latest), scripts specified through @snowpack/plugin-run-script run after snowpack builds.

So I see errors like: [snowpack] config.mount[~/Desktop/11st-starter-kit/src/_site]: mounted directory does not exist.

This seems to introduced in 3.0.13 (See https://github.com/stefanfrede/11st-starter-kit/issues/156). Although same error message exists on 3.0.12, build seems to work as intended (scripts build before snowpack).

To Reproduce

git clone https://github.com/stefanfrede/11st-starter-kit.git
cd 11st-starter-kit
git checkout fa11f234e5beb705eaed1752a3047c24a2d8221f
npm install
npm install snowpack@latest
npx snowpack --version
npx snowpack build

See how snowpack builds before eleventy does:

[18:23:24] [snowpack] 3.3.6
[18:23:26] [snowpack] config.mount[~/Desktop/11st-starter-kit/src/_site]: mounted directory does not exist.
-------------------------------------------------------------------------------------------------------------
[18:23:26] [snowpack] ! building files...
-------------------------------------------------------------------------------------------------------------
[18:23:27] [eleventy] Writing src/_site/404.html from ./src/404.md.
[18:23:27] [eleventy] Writing src/_site/about/index.html from ./src/about.md.
[18:23:27] [eleventy] Writing src/_site/index.html from ./src/index.md.
[18:23:27] [eleventy] Copied 16 files / Wrote 3 files in 0.33 seconds (v0.12.1)
[18:23:27] [eleventy] Command completed.
-------------------------------------------------------------------------------------------------------------
[18:23:28] [snowpack] import alpinejs@latest → https://pkg.snowpack.dev/alpinejs
[18:23:28] [snowpack] pin dependency to this version: `snowpack add alpinejs`
[18:23:29] [snowpack] ✔ files built. [2.94s]
[18:23:29] [snowpack] ! building dependencies...
[18:23:29] [snowpack] ✔ dependencies built. [0.00s]
[18:23:29] [snowpack] ! writing to disk...
[18:23:29] [snowpack] ✔ write complete. [0.03s]
[18:23:29] [snowpack] ! optimizing build...
.......

Relevent snowpack config:

  plugins: [
    [
      '@snowpack/plugin-run-script',
      {
        cmd: 'eleventy',
        watch: '$1 --watch',
      },
    ],
  ],

Expected behavior

Scripts build before snowpack does. Here (3.0.12) builds fine, although with same error message.

❯ npx snowpack --version
[snowpack] 3.0.12
❯ npx snowpack build
[snowpack] config.mount[~/Desktop/11st-starter-kit/src/_site]: mounted directory does not exist.
[eleventy] Writing src/_site/404.html from ./src/404.md.
[eleventy] Writing src/_site/about/index.html from ./src/about.md.
[eleventy] Writing src/_site/index.html from ./src/index.md.
[eleventy] Copied 16 files / Wrote 3 files in 0.23 seconds (v0.12.1)
[snowpack] ! building source files...
[snowpack] ✔ build complete [1.21s]
[snowpack] ! building dependencies...
[snowpack] ✔ dependencies ready! [1.17s]
[snowpack] ! verifying build...
[snowpack] ✔ verification complete [0.00s]
[snowpack] ! writing build to disk...
[snowpack] ! optimizing build...
[snowpack] ✔ optimize complete [0.15s]
[snowpack] ▶ Build Complete!

Related

2936

[jalovatt]

Is your build actually breaking?

That warning is because your build script deletes src/_site before running the Snowpack, so Snowpack rightfully can't find it, but that's happening at the "load the config and make sure it looks okay" step. Eleventy is still running before any of the actual build stuff.

Changing the clean script to rimraf src/_site/* gives me no warning, a completed build, and files from _site making it into the build folder.

[amithm7]

@jalovatt src/_site/ is output of 11ty, not snowpack; why would it be deleted? It doesn't. Snowpack outputs to dist/ which is empty to begin with.

Could you check the commands in To Reproduce section.

[jalovatt]

Look in your package.json - npm run build runs clean first, which deletes src/_site so Eleventy gets a fresh folder to build into.

https://github.com/stefanfrede/11st-starter-kit/blob/2896d06cae325d04010193b419d2e58a74375245/package.json#L41-L48

[amithm7]

@jalovatt I'm not refering to the latest repo. And neither am I using npm run build.

Could you check the commands in To Reproduce section. :pray:

[jalovatt]

Sorry, I didn't see that you were using npx snowpack build. Regardless, I only get that warning on the first build, because that folder correctly does not exist yet.

mounted directory does not exist is just a warning when Snowpack starts. Nothing is breaking, at least here. The build runs fine, and as far as I can tell the output is correct - /dist has the files from _site in it.

Perhaps you (or maybe me?) are misunderstanding the order Snowpack is doing things in? Just in case:

1. It starts up, takes in the config you gave it, and gives that warning because you told it to mount a folder that isn't there.
2. Eleventy runs.
3. Snowpack builds. src/_site is there now, so everything should be fine.

[colelawrence]

I'm seeing a similar issue, where I can acknowledge that Snowpack now scans all mounted directories up front, before any plugins are able to make changes.

In my case, I'm trying to create a WASMPack plugin for Snowpack, but during snowpack build, the fileToUrlMapping is scanned and filled out before wasm-pack has an opportunity to even run.

This is a difference in behavior from snowpack 2, where there must not have been any sort of mapping. So, what ends up happening is that Snowpack can't find any files that were generated during the run step. This seems very likely similar to what @amithm7 is experiencing, and I hope this issue is addressed soon, else we're stuck on snowpack 2, and eventually will need to find an alternative to snowpack 2 if this functionality is not added to snowpack 3.

I would really appreciate another contributor to join this conversation who can talk to these changes and offer ideas towards building for this use case.

Alternatively, we'll have to run our build steps sequentially, first running a wasm-pack command, then running snowpack build, which is far slower since it's no longer able to be ran concurrently (again, because snowpack starts off by creating a cache of the crawled directories in the fileToUrlMapping OneToMany map).

[gremo]

I have the exact same problem. Latest working version is 3.0.13. My Eleventy _out directory is full of HTML files, the plugin run... but HTML files are missing from Snowpack _dist folder.

It's so sad that something has changed. This official documentation MUST be updated because it's not valid anymore.

[jungaretti]

@amithm7 I am having the exact same issue as you, src/_site and all! The whole reason I'm using @snowpack/plugin-run-script is to build that directory with Eleventy before building my whole site with Snowpack.

Sanity Check

I replaced my Eleventy build command with a simple echo. Check out relevant Snowpack config:

  plugins: [
    [
      "@snowpack/plugin-run-script",
      {
        cmd: "echo Do you see me?", // production build command
      },
    ],
  ],

This seems to work exactly as intended. Snowpack runs my command before building:

> jungaretti.com@1.0.0 build:snowpack /Users/jungaretti/Repos/jungaretti.com
> snowpack build

[15:21:31] [echo] Do you see me?
[15:21:31] [echo] Command completed.
[15:21:31] [snowpack] ! building files...
[15:21:31] [snowpack] ✔ files built. [0.02s]
[15:21:31] [snowpack] ! building dependencies...
[15:21:31] [snowpack] ✔ dependencies built. [0.00s]
[15:21:31] [snowpack] ! writing to disk...
[15:21:31] [snowpack] ✔ write complete. [0.00s]
[15:21:31] [snowpack] ▶ Build Complete!

Back to Eleventy

Here's my desired Snowpack config:

  plugins: [
    [
      "@snowpack/plugin-run-script",
      {
        cmd: "eleventy", // production build command
        watch: "eleventy --watch", // dev server command
      },
    ],
  ],

It seems like Snowpack doesn't even run this command:

> jungaretti.com@1.0.0 build:snowpack /Users/jungaretti/Repos/jungaretti.com
> snowpack build

[15:29:13] [snowpack] ! building files...
[15:29:13] [snowpack] ✔ files built. [0.02s]
[15:29:13] [snowpack] ! building dependencies...
[15:29:13] [snowpack] ✔ dependencies built. [0.00s]
[15:29:13] [snowpack] ! writing to disk...
[15:29:13] [snowpack] ✔ write complete. [0.00s]
[15:29:13] [snowpack] ▶ Build Complete!

Workaround/Hack

Do something like this in package.json:

  "scripts": {
    "build:eleventy": "eleventy",
    "build:snowpack": "snowpack build",
    "build": "npm-run-all build:*",
    "dev:snowpack": "snowpack dev",
    "dev": "npm-run-all build:eleventy dev:snowpack"
  },

Instead of relying on Snowpack to run Eleventy and generate src/_site, I use npm-run-all to do an Eleventy build before my Snowpack build. This is pretty dumb, since it's (theoretically) doing the same work twice, but it gets the job done for now. I hope this issue is fixed soon!

I agree with @gremo that we should update the docs to reflect the fact that commands are NOT always run before Snowpack builds, as https://github.com/snowpackjs/snowpack/tree/main/plugins/plugin-run-script#readme claims they are...

[andreasphil]

This is probably the same bug I have observed here: https://github.com/snowpackjs/snowpack/issues/3095

I started working on a fix for snowpack build back then: https://github.com/andreasphil/snowpack/tree/build-step-plugin-order

I made snowpack await plugins during builds and delayed mounting until the plugins finished, so all plugin output is included and up to date. That solves the issue for me, but it breaks a couple of tests and I don't know when I'll get around to fixing them - life has been busy lately 😶 (if anyone else has the time feel free to take it from here).

Regarding snowpack dev, that still only works if at least some eleventy output already exists when the dev server boots. Apparently the eleventy plugin starts after the initial mounting but finishes before file watching begins. Somehow that causes snowpack to not notice the output files at all, even if they change again, until I restart the server. After that it works fine though, so it's not a huge problem in development except when serving a repository for the first time after cloning or rming the eleventy output. In the interest of keeping the dev server startup fast I don't think we want to await all plugins like in snowpack build. So I'm not sure what a good solution would be here, but probably the plugin output should get picked up by the file system watcher just like it does with regular JS files.

@colelawrence Someone in the other ticket noted that everything worked as expected until v3.0.13 - you could give that a try if you need something more recent than 2?

[andreasphil]

Another thing: From the docs @gremo linked (emphasis mine) -

This plugin allows you to run any CLI command as a part of your dev and build workflow. This plugin doesn't affect your build output, but it is useful for connecting developer tooling directly into Snowpack. Use this to add meaningful feedback to your dev console as you type, like TypeScript type-checking and ESLint lint errors. This doesn't affect how Snowpack builds your site.

This makes me wonder whether what snowpack does is indeed the intended behavior and we're just not meant to run eleventy as a plugin command. On the other hand, the plugin-run-script readme clearly says "will run before Snowpack builds" and "production build command" 🤔

[colelawrence]

I don't have capacity at the moment to test and troubleshoot as we several custom snowpack 2 plugins that I'm pretty sure will be broken for other reasons.

If there is a resolve to this issue and the other issue I have with Dev server custom headers for COOP & COEP, then I'll be interested in debugging. I'd also be interested if someone familiar with Snowpack's codebase was able to pair with me on the issue, here.