SSL errors when using `gatsby develop --https`

[patrickarlt]

Description

When running gatsby develop --https both with without --cert-file/--key-file SSL does not work in Firefox, Chrome or Safari (screenshots below).

Steps to reproduce

1. I've created a minimal reproduction in https://github.com/patrickarlt/gatsby-ssl-issue-reproduction. This is based off the default starter with the latest version of Gatsby.
2. npm install
3. npm run start:devcert this starts the local dev server and automatically creates SSL certs.
4. Observe failures.
5. Install mkcert, for Mac this means:
   • brew install mkcert
   • brew install nss
   • mkcert install
   • mkcert localhost
6. npm run start:local-certs this starts the dev server with the local certs from mkcert
7. Observe failures.

In order to check that the certs are valid you can use http-server:

• npm i -g http-server
• http-server . -C localhost.pem -K localhost-key.pem -S -a localhost

This displays the directory structure of our project tin the browser over https://... note the lock icon in Chrome:

Expected result

The server should successfully serve pages over https://....

Actual result

Browsers produce SSL errors.

Environment

Gatsby

  System:
    OS: macOS Mojave 10.14.6
    CPU: (8) x64 Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz
    Shell: 5.3 - /bin/zsh
  Binaries:
    Node: 14.3.0 - ~/.nvm/versions/node/v14.3.0/bin/node
    npm: 6.14.5 - ~/.nvm/versions/node/v14.3.0/bin/npm
  Languages:
    Python: 2.7.10 - /usr/bin/python
  Browsers:
    Chrome: 81.0.4044.138
    Firefox: 77.0
    Safari: 13.1
  npmPackages:
    gatsby: ^2.22.3 => 2.22.3

OpenSSL

$ openssl version
LibreSSL 2.6.5

mkcert

$ brew info mkcert 
mkcert: stable 1.4.1 (bottled)
Simple tool to make locally trusted development certificates
https://github.com/FiloSottile/mkcert
/usr/local/Cellar/mkcert/1.4.1 (6 files, 5.3MB) *
  Poured from bottle on 2020-04-08 at 18:45:14
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/mkcert.rb
==> Dependencies
Build: go ✘
==> Analytics
install: 3,827 (30 days), 11,641 (90 days), 47,679 (365 days)
install-on-request: 3,351 (30 days), 10,186 (90 days), 40,390 (365 days)
build-error: 0 (30 days)

nss

$ brew info nss
nss: stable 3.52 (bottled)
Libraries for security-enabled client and server applications
https://developer.mozilla.org/docs/NSS
/usr/local/Cellar/nss/3.52 (225 files, 34.1MB) *
  Poured from bottle on 2020-05-19 at 16:52:58
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/nss.rb
==> Dependencies
Required: nspr ✔
==> Analytics
install: 38,963 (30 days), 105,774 (90 days), 394,028 (365 days)
install-on-request: 8,110 (30 days), 19,022 (90 days), 62,334 (365 days)
build-error: 0 (30 days)

[ascorbic]

Hi, Thanks for this report, and the great repro. I can also reproduce this locally myself, and it seems to be a regression in gatsby@2.22.0/gatsby-cli@2.12.26. I can't immediately work out what the problem is, but here are the results of my testing so far, running gatsby develop --https:

➜  openssl s_client -connect localhost:8000 -prexit
CONNECTED(00000005)
4613344876:error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number:/AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-47.100.4/libressl-2.8/ssl/ssl_pkt.c:386:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 5 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Start Time: 1590056545
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 5 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Start Time: 1590056545
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

➜  curl -vvv https://localhost:8000
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8000 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
* Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number

The error usually indicates that we are trying to connect to an HTTP server using HTTPS, but if I try that here it just hangs indefinitely. I'm unsure whether this behaviour would give the same error if attempting to connect via HTTPS. The server and listener instances are do not log any connect or error events.

I'm going to pass this on to @mxstbr who broke it!

[hendra-go]

Probably related to "proxy server with self signed certificate" ?

[mxstbr]

Yes this is related to #22759, which introduced a proxy server to the develop process. I'm planning to take a look at this tomorrow (OoO today), however if anybody needs this urgently the fix might be to pass the SSL options to http-proxy OR using the https module to create the server (or both?) in gatsby/utils/develop-proxy.ts.

[JustFly1984]

Confirming an issue - reverting to gatsby@2.21.40 helped, waiting for a fix

[rodrigo-arias]

Same thing here, localhost was not responding even without --https. Reverting to 2.21.xx worked

[JustFly1984]

Version 2.22.9 still failing tests. We are using cypress tests with https in our ci/cd, and dependabot tracking new versions of npm packages fails ci/cd tests with error:

success createPages - 0.082s
success createPagesStatefully - 0.037s
success onPreExtractQueries - 0.001s
success update schema - 0.028s
success extract queries from components - 0.381s
success write out requires - 0.033s
success write out redirect data - 0.008s
success onPostBootstrap - 0.017s
⠀
info bootstrap finished - 44.105s
⠀
success run queries - 0.163s - 37/37 227.51/s
warning 
  When setting `useBuiltIns: 'usage'`, polyfills are automatically imported when needed.
  Please remove the `import '@babel/polyfill'` call or use `useBuiltIns: 'entry'` instead.
⠀
You can now view pooltickets-gatsby in the browser.
⠀
  https://localhost:8000/
⠀
View GraphiQL, an in-browser IDE, to explore your site's data and schema
⠀
  https://localhost:8000/___graphql

Note that the development build is not optimized.
To create a production build, use gatsby build
⠀
success Building development bundle - 30.032s
Error: Timeout
    at MergeMapSubscriber.project (/home/runner/work/pooltickets-gatsby/pooltickets-gatsby/node_modules/wait-on/lib/wait-on.js:145:74)
    at MergeMapSubscriber._tryNext (/home/runner/work/pooltickets-gatsby/pooltickets-gatsby/node_modules/rxjs/internal/operators/mergeMap.js:69:27)
    at MergeMapSubscriber._next (/home/runner/work/pooltickets-gatsby/pooltickets-gatsby/node_modules/rxjs/internal/operators/mergeMap.js:59:18)
    at MergeMapSubscriber.Subscriber.next (/home/runner/work/pooltickets-gatsby/pooltickets-gatsby/node_modules/rxjs/internal/Subscriber.js:66:18)
    at AsyncAction.dispatch [as work] (/home/runner/work/pooltickets-gatsby/pooltickets-gatsby/node_modules/rxjs/internal/observable/timer.js:31:16)
    at AsyncAction._execute (/home/runner/work/pooltickets-gatsby/pooltickets-gatsby/node_modules/rxjs/internal/scheduler/AsyncAction.js:71:18)
    at AsyncAction.execute (/home/runner/work/pooltickets-gatsby/pooltickets-gatsby/node_modules/rxjs/internal/scheduler/AsyncAction.js:59:26)
    at AsyncScheduler.flush (/home/runner/work/pooltickets-gatsby/pooltickets-gatsby/node_modules/rxjs/internal/scheduler/AsyncScheduler.js:52:32)
    at listOnTimeout (internal/timers.js:549:17)
    at processTimers (internal/timers.js:492:7)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
##[error]Process completed with exit code 1.

[Undistraction]

An additional piece of info that might be useful:

I'm using the Netlify CMS plugin, and as well as the issue outlined above, I'm seeing an additional port being added to the CMS URL.

Usually a build will output the following:

Netlify CMS is running at https://example.test/admin/ However the build is now outputing: Netlify CMS is running at https://example.test/52845/admin/

Rolling back to gatsby@2.21.x fixes both issues. There appears to be no need to rollback gatsby-cli.

[JustFly1984]

2.22.12 still failing

[hendra-go]

2.22.12 still failing

I understand your frustration, but please be patience. @mxstbr already created a PR #24335, just wait until gatsby core team review and merge the changes.

[pieh]

We released gatsby@2.22.19 which should fix the problems (I couldn't reproduce problems anymore when testing it manually) - please try upgrading and let us know - if there are still issues please create new issue for it (it's easier for us to track it this way)

[rodrigo-arias]

Still not work for me, I'm using gatsby develop --host=0.0.0.0 The build runs normal, but the localhost does not respond

[JustFly1984]

@pieh thank you, issue is solved for us, both dev and prod and test working great, although there is one new issue #24758

[pieh]

Still not work for me, I'm using gatsby develop --host=0.0.0.0 The build runs normal, but the localhost does not respond

@rodrigo-arias I just opened https://github.com/gatsbyjs/gatsby/pull/24761 which should fix --host problem. Sorry for not checking this when I was reviewing --https related problems (but tbh this is related, but really different issue, given that --host=0.0.0.0 doesn't work even without using --https)

[rodrigo-arias]

@pieh you're right it's related but it's not the same. Thank you for opening the related ticket

[pieh]

@rodrigo-arias Just published gatsby@2.22.20 with the --host fix - if you could try upgrade again and let us know I would really appreciate it

[JustFly1984]

@rodrigo-arias @pieh can somebody please explain the use cases for --host and --host=0.0.0.0 specifically?

[pieh]

can somebody please explain the use cases for --host and --host=0.0.0.0 specifically?

If you don't use --host we will "bind" web server to localhost hostname, using 0.0.0.0 allows us to "bind" to every possible ip:

In the context of servers, 0.0.0.0 can mean "all IPv4 addresses on the local machine". If a host has two IP addresses, 192.168.1.1 and 10.1.2.1, and a server running on the host is configured to listen on 0.0.0.0, it will be reachable at both of those IP addresses.

In practice - this is a way to test sites on other devices in your network and not just and computer you develop on. I think most common case is to check your site on actual mobile device that is connecting to same network as your computer.

Other cases include running gatsby develop inside docker containers or vms in general where you want to make site available for preview on host machine etc.

[rodrigo-arias]

@pieh no actually, in this case I got an error before launching the localhost

[pieh]

@rodrigo-arias is this intentional that you can't resolve localhost? Do you run some specific setup?

Errors like that can't be found all over the place:

• https://github.com/webpack/webpack-dev-server/issues/420
• https://github.com/angular/angular-cli/issues/2227

And root cause of those is often something borked network stack config (tho removal of localhost host can be intentional, tho I don't know why you would do that)

[rodrigo-arias]

@pieh I thought it was related to a docker instance, but you were right about the localhost resolution, it didn't worked in any case. Just in case someone have this problem on Mac

Everything is working now. Thank you

[muuvmuuv]

Having this issue again in v2.23.3. I could reproduce it with the latest starter and certs generated through this script: https://github.com/m8finder/webssl

    "develop-ssl": "gatsby develop -H test.lcl -S -k certs/dev.key -c certs/dev.crt",

[JustFly1984]

@muuvmuuv for me it works in combination for 2.23.4 and pinning devcert: "1.1.0" in package.json on mac:

"resolutions": {
    "devcert": "1.1.0"
  },

with 2.23.7 it doesn't work anymore @pieh please reopen

[muuvmuuv]

Still happening with 2.23.8.

@JustFly1984 I am not using devcert.

[muuvmuuv]

• 2.23.4: no errors
• 2.23.5, 2.23.6:
  • ~Warning: Invalid arguments supplied to oneOf, expected an array, got 3 arguments. A common mistake is to write oneOf(x, y, z) instead of oneOf([x, y, z]).~
  • TypeError: Unexpected response from worker: undefined
  • no start at all
• >2.23.7:
  • ~Warning: Invalid arguments supplied to oneOf, expected an array, got 3 arguments. A common mistake is to write oneOf(x, y, z) instead of oneOf([x, y, z]).~
  • this starts the server but ERR_SSL_PROTOCOL_ERROR

[JustFly1984]

@pieh still same issue in @2.23.9

[muuvmuuv]

Found the issue: program.ssl isn't set before creating the proxy...

https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/commands/develop.ts#L121-L125

This PR should not have been merged! https://github.com/gatsbyjs/gatsby/pull/25015

[JustFly1984]

@pieh @wardpeet still hangs in 2.23.10, reverted back to 2.23.4

[muuvmuuv]

It isn't yet released @JustFly1984 . Just go to node_modules/gatsby/dist/commands/develop.js:127 and move the proxy and developProcess (together) below the program.https if-clause manually.

[JustFly1984]

@muuvmuuv can't do that in ci/cd workflow, waiting for release. maybe not closing an issue till fix released?

[muuvmuuv]

@JustFly1984 check the latest release. 2.23.11 should include the fix

[JustFly1984]

@muuvmuuv @pieh @wardpeet Thank you, it is fixed

[ProteanCode]

I just started to learn gatsby and this was bit confusing error since I got this on a new site made with gatsby cli tool. I installed the 2.23.11

Small recap: The gatsby develop --https still not work in 2.23.11 (the localhost is invalid domain) The gatsby develop --host=0.0.0.0 --https still not work in 2.23.11 (the localhost is invalid domain)

I had to manually create the pem files (using mkcert) and run: gatsby develop --https --key-file ./keys/localhost-key.pem --cert-file ./keys/localhost.pem

You may also need to specify --ca-file if your web browser does not trust it afaik

Since the docs are highly misleading at this point this issue should not be closed unless docs update or code fix.

Using Ubuntu 18.04 with Brave web browser

When gatsby cli will use the 2.23.11?

[JustFly1984]

@ProteanCode you need to pin devcert to 1.1.0 in package.json

[MitchMigala]

Has this issue been resolved in Gatsby 2.29.x ? I am getting This site can’t provide a secure connection localhost sent an invalid response. ERR_SSL_PROTOCOL_ERROR Still when trying to use --https. I have a docker container running nginx for the front end as a reverse proxy to a node js server. I have nginx setup to listen to port 443 and am telling nginx and gatsby to use the ssl certificates. This issue is killing me. Anyone have any ideas?

[mSnus]

Still facing the same problem in 2022 with gatsby 4.13.0

Was trying to access my app in dev mode via HTTPS on my real domain, but receive SSL errors no matter what certs I use.

I need real domain for cookie authorization on back-end (Laravel Sanctum).