Varying-Vagrant-Vagrants / VVV

An open source Vagrant configuration for developing with WordPress
https://varyingvagrantvagrants.org
MIT License
4.54k stars 847 forks source link

No VVV Self Signed Certifications Will Validate #1411

Closed AramZS closed 6 years ago

AramZS commented 6 years ago

Expected Behavior

VVV should be accessible either by HTTP or by valid self-signed HTTPS certificate. If neither is available the suitable instructions to self-sign a certificate from inside the VVV box should be made available.

Current Behavior

screenshot 2018-02-01 00 30 55

For unclear reasons the parsing of the cert has changed in Opera, Firefox and Chrome. Firefox allows a bypass by user input. Chrome allows a bypass by typing badidea and pressing enter while in the warning screen. Opera has no bypass that I can tell. Marking Always Trust on the certs in Apple Keychain doesn't help. I've tried 4 different ways of generating self-signing certificates now, including two that have worked previously in VVV and in production environments.

I realize this was in #1388, but I don't believe that a wontfix is satisfactory. VVV is currently appearing unusable on three major browsers, two of which have no clear bypass solution. Arguably this is a breaking error. If the solution is to tell users to use Firefox and accept the certification bypass, that should be documented clearly in the setup process.

Possible Solutions

I believe a solution must be available, as I do not have these same issues running https via Node.

Steps to Reproduce (for bugs)

  1. Pull down a new copy of VVV from Github
  2. Boot up machine.
  3. Navigate to any of its URLs via Opera, Firefox or Chrome.

Context

Attempting to solve this problem has take up extensive amounts of time.

None of the previously posted solutions in VVV or on common help sites work.

A lack of HTTPS makes building secure applications on top of WordPress (like ones that send callbacks via HTTP Request) impossible to test in VVV.

Furthermore we are constantly trying to educate users and developers in the need to bring HTTPS to the web. A lack of support in the development tool counteracts work being done in this regard in the community.

Your Environment

I would be glad to help test or walk through possible solutions here, but at the very least the problem should be documented so as not to confuse new users.

tomjn commented 6 years ago

You shouldn't be using .dev, it's a Google TLD and has special restrictions imposed on it by chromium based browsers. VVV provides .test varieties of all the sites

Also, VVV doesn't support https out of the box. You can configure and use your own certificate authority, but there's no documentation on how to do that, and the instructions are different for each platform. If you'd like to contribute the details of how to do so I can tidy it up and add it to the documentation

Documentation should be updated to reflect VVV is permanently inaccessible on these browsers.

This is only for the .dev TLDs, VVV switched to .test last year, e.g. http://vvv.test, and all new installs use .test. Auto-migration of old installs isn't possible as plugins do weird and wonderful things, but it is documented on the VVV site

VVV should never boot to HTTPS

It doesn't unless you specify explicitly, except for .dev domains, which is Chrome, that's a Google policy.

The certificates should be handled properly and in a way that they are accepted by common browsers.

VVV only sets up a single self signed certificate, but that's not because VVV supports self signed certificates, that's for historical reasons, and tbh it should be removed not fixed. That certificate only applies to the main VVV dashboard anyway

Documentation should be provided to allow users to generate valid SSL certification on their local machines.

Agreed, and while I wrote most of the docs, I can't document what I don't know. E.g. the HyperV docs I wrote are unmerged as I need a HyperV user to run through them and test, which I can't do on my Macbook.

What's more, we'd probably wan't to tear out all the HTTPS stuff and build it fresh anyway so it ran off of vvv-custom.yml, right now you'd have to fork the custom site template and modify the nginx config. We've also deprecated the default site in favour of a custom site template instance at the same URL

VVV is currently appearing unusable on three major browsers, two of which have no clear bypass solution.

The solution is don't use .dev TLDs, and switch to .test, which is protected by an RFC and not owned by Google. Continuing with .dev domains will only lead to a world of pain, and VVV has moved away from it for good reason. Even if VVV came with its own SSL authority, even then there is no guarantee .dev would work. Google owns it, and Google have placed restrictions on its use.


This is an open source project, for which there are a handful of people who work on it. Not all of their skills are the same, and there may be only a single person who can understand the entire scope of the project. Myself I understand a lot of it but I'm not a Vagrant expert, and the other providers such as VMWare or Parallels are foreign territory to me.

Ideally, you would specify an SSL certificate in vvv-custom.yml, or say ssl: true and place a certificate in the provision folder probably named vvv-ssl.cert or whatever. If you'd like to build that you're welcome to do so.

Trust me when I say those working on VVV want SSL to be easy. There are no docs on how to do SSL though because nobody who knows how has written any. There's no nice and easy feature for specifying certificates because nobody has built it. If you really need those, I strongly suggest you start a feature branch and a pull request.

As for your efforts to get SSL working, i don't know how you generated the certificates, how you told your browser they were valid, or how you got VVV to serve them. For all I know you're missing the directive in vvv-nginx.conf telling Nginx to use them, I don't know. So I can only guess as to why it isn't working for you, due to a lack of information, and missing knowledge. I know other people have managed it though, so it's not a universal thing

AramZS commented 6 years ago

So for the sake of reference, I used the processes documented at, which generated certs successfully and I placed the path to the generated certs in each case in vvv-nginx.conf though I didn't try anything in regard to the provision folder, I generated it in the running machine, accessed via vagrant ssh:

The fourth process I attempted didn't even generate a cert so it's not worth mentioning the details of.

I took down and up the machine in each case and got the same results.

The idea of using .test definitely sounds like part of the solution, but I vagrant destroyed both the old machine and the new machine, wiped out the old version's files, pulled down the latest instance of VVV, and built it from scratch, the machines continue to redirect to .dev versions of the domain in both Chrome and Opera (though I guess it doesn't in Firefox oddly). Something seems to be occurring pushing these browsers to direct to dev? VVV.local seems to work though. Perhaps this is because I hit the WordPress setup at local.wordpress.dev and so mapped that domain to the WordPress database? It might make sense to force new versions to redirect to .test versions of the site to avoid this?

tomjn commented 6 years ago

When you say clearing it out, I assume you're destroying the entire VVV folder? Including the www folder and vvv-config.yml or vvv-custom.yml? As well as any SQL backups created by Vagrant triggers? If you just run vagrant destroy and recreate the machine, it'll create the sites out of vvv-custom.yml or vvv-config.yml, and it'll also restore the databases from the vagrant triggers backups, restoring any databases from previous sites that used .dev as the primary site URL

For that scenario, either a WP CLI search replace is required to change the domain, the same way any site migration with a URL change would, or, completely delete the VVV folder after vagrant destroy and do a fresh git clone


Re: the wiki, it's there for legacy reasons, nothing in the wiki has been tested with VVV 2, so unless it's on the main VVV website I'd take it with a pinch of salt. Because there's stuff in there that has no equivalent in the site it hasn't been deleted, but it also needs verifying and checking

AramZS commented 6 years ago

Yeah, I vagrant destroyed then chucked the entire VVV folder into the trash, pulled down a clean version from git and started from scratch. I've done it a second time to see if the problem persists if I start with WordPress on .local.

Anyway we could add a notice to the wiki? I understand the need to leave it in for legacy reasons, but it beats out the VVV website in search results pretty consistently.

tomjn commented 6 years ago

eh .local has issues, in that it conflicts with Apple machines that use bonjour, eg. does vvv.local refer to the VVV dashboard, or an iMac named VVV? Always use .test

Anyway we could add a notice to the wiki? I understand the need to leave it in for legacy reasons, but it beats out the VVV website in search results pretty consistently.

There's a note at the top of the homepage, but there's no way to add a banner to every single page without editing every single page

tomjn commented 6 years ago

Current master has this for the default site:

  # The wordpress-default configuration provides an installation of the
  # latest version of WordPress.
  wordpress-default:
    repo: https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git
    hosts:
      - local.wordpress.test

There should be no .local version of that site, only .test. If local.wordpress.local works for you with a fresh copy of VVV, then something somewhere has persisted or been modified

AramZS commented 6 years ago

Sorry, I meant .test there

AramZS commented 6 years ago

Ok, I can confirm that a clean install, going to the .test domain succeeds in a properly set up site. For those who work off of unclear documentation by accident, can we attempt to assure that the .dev domain is always redirected to .test?

Also, if I can build a proper SSL cert in my system, and get the configuration running, is there any reason I couldn't just commit it within the site folder? If it works with an sh script should I be setting that up in the provision folder. I'd be glad to contribute back a working process, once I get it working. What would be the best approach, docs, a cert committed into the repo, or a script to execute as part of the provisioning process?

tomjn commented 6 years ago

That would break some users who refuse to migrate, e.g. you can still use .dev with http despite the errors if you type something into the error window on chrome, it used to be badidea, though they changed it once they enforced the HSTS on that TLD.

For existing users, we added docs for migration, and put banners across the site entries in the dashboard warning them to switch to .test with a link to the docs

Also, if I can build a proper SSL cert in my system, and get the configuration running, is there any reason I couldn't just commit it within the site folder? If it works with an sh script should I be setting that up in the provision folder. I'd be glad to contribute back a working process, once I get it working. What would be the best approach, docs, a cert committed into the repo, or a script to execute as part of the provisioning process?

I think the most helpful thing right now is a set of instructions that can be turned into docs. I'm mindful that if a certificate authority is setup inside the VM then it'll be destroyed along with the VM and you'd have to redo all the certificates again on the next provision. First step of actual implementation should be specifying the certificate file in the provision folder and making sure the nginx config and the dashboard pick that up

AramZS commented 6 years ago

So I've run into an issue. There is a chunk in provision.sh that rewrites the key on every provision and sets it as the base level key in the nginx files, overwriting configurations that occur at site level vvv-nginx.conf files. It looks like:

  # Create an SSL key and certificate for HTTPS support.
  if [[ ! -e /etc/nginx/server-2.1.0.key ]]; then
      echo "Generate Nginx server private key..."
      vvvgenrsa="$(openssl genrsa -out /etc/nginx/server-2.1.0.key 2048 2>&1)"
      echo "$vvvgenrsa"
  fi
  if [[ ! -e /etc/nginx/server-2.1.0.crt ]]; then
      echo "Sign the certificate using the above private key..."
      vvvsigncert="$(openssl req -new -x509 \
            -key /etc/nginx/server-2.1.0.key \
            -out /etc/nginx/server-2.1.0.crt \
            -days 3650 \
            -subj /CN=*.wordpress-develop.test/CN=*.wordpress.test/CN=*.wordpress-develop.dev/CN=*.wordpress.dev/CN=*.vvv.dev/CN=*.vvv.local/CN=*.vvv.localhost/CN=*.vvv.test 2>&1)"
      echo "$vvvsigncert"
  fi

What is the best way to remove, amend, or allow site-level overrides of this directive?

AramZS commented 6 years ago

Ok... weird but true, activating this plugin - https://wordpress.org/plugins/really-simple-ssl/ seems to have fixed the above issue.

If I follow the instructions at https://github.com/Varying-Vagrant-Vagrants/VVV/wiki/Site-specific-self-signed-SSL-certificates (with a modification of where the ssl folder lives, it should be one step up from the directory the conf file is in) and then install that plugin on the .test. sites it seems to be working.

AramZS commented 6 years ago

(Also it looks like restarting nginx isn't enough. A full vagrant provision is needed.)

tomjn commented 6 years ago

I've gone through the wiki and added notices to more of the pages, and eliminated most if not all instances of .dev TLDs from examples

Mte90 commented 6 years ago

Do you can do a complete guide about that for VVV2? I have legacy stuff and is not ready to migrate to a different TLD but migrate to https is not a big problem.

tomjn commented 6 years ago

I don't have any guides on setting up https, it's something being looked into, but there is no recommended article or incomplete instructions I can direct you to.

tomjn commented 6 years ago

There's now a tls-ca utility that adds a certificate authority and generates certificates, it's in beta so needs more testing but should suffice, WIP docs at https://github.com/Varying-Vagrant-Vagrants/varyingvagrantvagrants.org/pull/73

tomjn commented 6 years ago

As an aside, .dev will never work with https short of getting hired by Google and having your .dev domain officially added. Nothing can be done, and browser restrictions appear to be tightening

lock[bot] commented 4 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.