Errors in deployment pipeline not treated as fatal

[jeffposnick]

I just had a bad experience during my first attempt at running npm run deploy locally, and I wanted to document the errors first, and then think about how we can change the deployment pipeline to prevent it from happening again.

Here's what I was seeing:

jeffy [~/git/workbox-microsite]$ npm run deploy

> workbox-microsite@1.0.0 deploy /Users/jeffy/git/workbox-microsite
> npm run test && firebase deploy

> workbox-microsite@1.0.0 test /Users/jeffy/git/workbox-microsite
> npm run lint && gulp build && mocha ./test/node/**/*.js

> workbox-microsite@1.0.0 lint /Users/jeffy/git/workbox-microsite
> eslint '.'

[13:17:25] Using gulpfile ~/git/workbox-microsite/gulpfile.js
[13:17:25] Starting 'build'...
[13:17:25] Starting 'clean'...
[13:17:25] Finished 'clean' after 302 ms
[13:17:25] Starting 'ref-docs'...

      If you want to build the latest refernce docs
      please run serve with the "--code" flag, passing
      in the path to the workbox repo.

          gulp serve --code ../workbox/

    Skipping build reference docs for v0.0.1
    Skipping build reference docs for v0.0.2
    Skipping build reference docs for v0.0.3
    Skipping build reference docs for v1.0.0
    Skipping build reference docs for v1.0.1
    Skipping build reference docs for v1.1.0
    Building reference docs for v1.2.0
    Downloading tagged release: v1.2.0
Cloning into '/Users/jeffy/git/workbox-microsite/tmp-1501521444992/v1.2.0'...
warning: redirecting to https://github.com/GoogleChrome/workbox.git/
remote: Counting objects: 450, done.
remote: Compressing objects: 100% (388/388), done.
remote: Total 450 (delta 84), reused 153 (delta 34), pack-reused 0
Receiving objects: 100% (450/450), 330.85 KiB | 0 bytes/s, done.
Resolving deltas: 100% (84/84), done.
Note: checking out 'ccd7c159eb5098e4296df6f3ce18298d9ecc011e'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

    Building JSDocs @ '/Users/jeffy/git/workbox-microsite/tmp-1501521444992/v1.2.0'. Version: v1.2.0
        JSDoc path: /Users/jeffy/git/workbox-microsite/node_modules/.bin/jsdoc
        JSDoc params: [
  "-c",
  "/Users/jeffy/git/workbox-microsite/jsdoc.conf",
  "--template",
  "/Users/jeffy/git/workbox-microsite/src/themes/_jsdoc",
  "-d",
  "/Users/jeffy/git/workbox-microsite/src/reference-docs/v1.2.0"
]
[13:17:28] Finished 'ref-docs' after 2.43 s
[13:17:28] Starting 'jekyll:build'...
[13:17:28] Starting 'npm-dependencies'...
[13:17:28] Starting 'clean-third-party'...
[13:17:28] Finished 'clean-third-party' after 1.05 ms
[13:17:28] Starting '<anonymous>'...
[13:17:28] Finished '<anonymous>' after 29 ms
[13:17:28] Finished 'npm-dependencies' after 30 ms
[13:17:28] Starting '<anonymous>'...
WARN: Unresolved specs during Gem::Specification.reset:
      listen (< 3.1, ~> 3.0)
      rb-fsevent (>= 0.9.4, ~> 0.9)
      rb-inotify (>= 0.9.7, ~> 0.9)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
/Users/jeffy/homebrew/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/spec_set.rb:87:in `block in materialize': Could not find sass-listen-3.0.7 in any of the sources (Bundler::GemNotFound)
        from /Users/jeffy/homebrew/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/spec_set.rb:80:in `map!'
        from /Users/jeffy/homebrew/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/spec_set.rb:80:in `materialize'
        from /Users/jeffy/homebrew/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/definition.rb:176:in `specs'
        from /Users/jeffy/homebrew/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/definition.rb:235:in `specs_for'
        from /Users/jeffy/homebrew/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/definition.rb:224:in `requested_specs'
        from /Users/jeffy/homebrew/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:118:in `block in definition_method'
        from /Users/jeffy/homebrew/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:19:in `setup'
        from /Users/jeffy/homebrew/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler.rb:100:in `setup'
        from /Users/jeffy/homebrew/lib/ruby/gems/2.4.0/gems/jekyll-3.4.5/lib/jekyll/plugin_manager.rb:36:in `require_from_bundler'
        from /Users/jeffy/homebrew/lib/ruby/gems/2.4.0/gems/jekyll-3.4.5/exe/jekyll:9:in `<top (required)>'
        from /Users/jeffy/homebrew/bin/jekyll:22:in `load'
        from /Users/jeffy/homebrew/bin/jekyll:22:in `<main>'
[13:17:29] Finished '<anonymous>' after 1.38 s
[13:17:29] Finished 'jekyll:build' after 1.41 s
[13:17:29] Starting 'styles'...
[13:17:29] Starting 'html'...
[13:17:29] Starting 'images'...
[13:17:29] Starting 'scripts'...
[13:17:29] Starting 'extras'...
[13:17:29] Starting 'images:copy'...
[13:17:29] Starting 'images:minified'...
[13:17:29] gulp-imagemin: Minified 0 images
[13:17:29] Finished 'html' after 243 ms
[13:17:29] Finished 'scripts' after 243 ms
[13:17:29] Finished 'extras' after 244 ms
[13:17:29] Finished 'images:copy' after 12 ms
[13:17:29] Finished 'styles' after 246 ms
[13:17:29] Finished 'images:minified' after 13 ms
[13:17:29] Finished 'images' after 246 ms
[13:17:29] Starting 'sw'...
[13:17:29] Finished 'sw' after 14 ms
[13:17:29] Finished 'build' after 4.4 s

  Check for Broken Links
    ✓ should have no broken links (55ms)

  1 passing (84ms)

=== Deploying to 'workbox-bab1f'...

i  deploying hosting
i  hosting: preparing build-prod directory for upload...
⚠  Warning: Public directory does not contain index.html
✔  hosting: 2 files uploaded successfully
i  starting release process (may take several minutes)...

✔  Deploy complete!

I'm assuming that the build failed due to that

Could not find sass-listen-3.0.7 in any of the sources (Bundler::GemNotFound)

error. Ideally, a failure in that step of the build pipeline should be treated as a fatal error, and stopping the chain. Failing that, the Firebase deploy step ideally would have treated the

⚠  Warning: Public directory does not contain index.html

as a fatal condition, and also refused to deploy.

[jeffposnick]

If a gulp task returns a rejected promise, and that promise rejection doesn't include a rejection value, then gulp won't consider that an error.

Which... I know ☹️

I actually bumped up into this previously, and filed https://github.com/gulpjs/gulp/issues/1642, and was told that it was fixed in gulp 4. But it's still not fixed with

$ gulp -v
[14:31:53] CLI version 1.3.0
[14:31:53] Local version 4.0.0-alpha.2

I'm going to follow-up with my original bug against gulp, but in the meantime, I'll file a PR to reject those promises with a value so that the build pipeline will properly terminate.