search

Varying-Vagrant-Vagrants / VVV

An open source Vagrant configuration for developing with WordPress
https://varyingvagrantvagrants.org
MIT License
4.54k stars 847 forks source link

General VVV 3 Testing #1792

Closed tomjn closed 5 years ago

tomjn commented 5 years ago 
__ __ __ __
\ V\ V\ V / A message from the VVV team
 \_/\_/\_/

VVV 3 is almost upon us!

Before it's merged into master, we need testers to confirm everything is as we think it is.

What Happened?

VVV 2 ran on Ubuntu 14, with PHP packages from Ondrej. Last week, Ubuntu 14 reached end of support, and Ondrej deleted the PHP packages as planned. We thought we had a mirror of the PHP packages, but it was incomplete ☹️.

So, we took a pull request to update VVV to Ubuntu 18, switched it to 18.04 LTS Bionic, and set about making the necessary changes.

What Changed in VVV 3?

The big changes first:

This means that to update, you must back up your database, destroy the VM, update, then reprovision. Important: Destroying and reprovisioning will erase the database.

Other changes:

Known Issues

Database Backups

VVV by default takes backups when you run vagrant halt/suspend, so check your database/backups folder. In future these will be in database/data/backups.

If you've turned off database backups, or haven't turned off your VM in a while, take the following steps:

  1. downgrade back to VVV 2: git fetch --tags && git checkout 2.6.0
  2. turn on the VM but dont provision: vagrant up
  3. run the backup DB script: vagrant ssh -c "db_backup"
  4. turn off the VM: vagrant halt
  5. return to VVV 3+: git checkout develop
  6. you can now update your VM to VVV 3, see the instructions below

Updating Your VM To VVV 3+

If you're happy and have your database files, you can update your VM to VVV 3+ with these commands:

  1. destroy the VM: vagrant destroy
  2. switch to the develop branch vagrant checkout develop && git pull
  3. provision a new VM: vagrant up --provision
  4. optionally restore the DB backups: vagrant ssh -c "db_restore"

Next Steps

Thanks for helping out and being patient, it's been a super tough week for myself and our contributors. If you're testing, let us know if it worked! If it didn't, feel free to comment below or in the relevant github issue.

If all is good you should see this at the end of provisioning:

    default: Running: inline script
    default: Post Vagrant provision script
    default: Restarting Nginx and MySQL
    default: 
    default:          yay  
    default:    ▄▀▀▀▄▄▄▄▄▄▄▀▀▀▄                __ __ __ __
    default:    █▒▒░░░░░░░░░▒▒█    Thanks for  \ V\ V\ V /
    default:     █░░█░░░░░█░░█     using        \_/\_/\_/
    default:  ▄▄  █░░░▀█▀░░░█  ▄▄ 
    default: █░░█ ▀▄░░░░░░░▄▀ █░░█ Provisioning has finished! Visit http://vvv.test

And you should be able to visit http://vvv.test and use your existing sites.

If all is bad, put a copy of the provisioner output in a gist at https://gist.github.com and share it with us, don't forget to include the VVV splash! it has important info such as your OS, Vagrant and VirtualBox versions

Lets get this shipped!

tomjn commented 5 years ago

I've had it in the past, if we know the database doesn't exist why not run a query beforehand to conditionally create it? We might already have something that tries that though, I'll have to check

tomjn commented 5 years ago

@cagross if you go into database/sql/import-sql.sh, and add these lines just after where it says pre_dot=${file%%.sql}:

    mysql -u root --password=root -e "CREATE DATABASE IF NOT EXISTS ${pre_dot}"
    mysql -u root --password=root -e "GRANT ALL PRIVILEGES ON ${pre_dot}.* TO wp@localhost IDENTIFIED BY 'wp';"

Then try it again, does it work as expected?

For reference, that section should look like this if you did the edit correct:

    do
    pre_dot=${file%%.sql}

    mysql -u root --password=root -e "CREATE DATABASE IF NOT EXISTS ${pre_dot}"
    mysql -u root --password=root -e "GRANT ALL PRIVILEGES ON ${pre_dot}.* TO wp@localhost IDENTIFIED BY 'wp';"

    mysql_cmd='SHOW TABLES FROM `'$pre_dot'`' # Required to support hypens in database names
    db_exist=`mysql -u root -proot --skip-column-names -e "$mysql_cmd"`
    if [ "$?" != "0" ]
cagross commented 5 years ago

@tomjn OK I'll try. But now vagrant up has gone back to the VirtualBox error I was having earlier. Going to try a reboot :-/

tomjn commented 5 years ago

If it helps, you can vagrant suspend and vagrant resume instead of and up/halt, it's the equivalent of closing a lid on a laptop rather than powering down and wakes the VM up much faster

tomjn commented 5 years ago

@cagross as an aside, are you running any of these commands with sudo? I remembered encountering that problem with somebody at WCEU contributor day, and it turned out they'd used sudo vagrant up etc which lead to problems

cagross commented 5 years ago

Well, I can't get past this VirtualBox error now. I can give it another shot tomorrow. But this is pretty deflating.

are you running any of these commands with sudo?

No I am not.

coderkevin commented 5 years ago

yes, u should be using develop, so if you haven't done git pull recently, you may want to do so and do vagrant destroy as well. since the develop branch is using the latest ubuntu 18.04

Yeah, that's what I did. A destroy was the only way to clear out the bad VM state from master.

Master will work once VVV 3 is merged, it just needs to finish the release process, the release post has been written so I'll merge and tag it later today

If it's going to be more than a day or so before develop gets merged into master, I'd humbly suggest at least updating the readme (and maybe docs) to have folks check out the develop branch so they don't waste hours trying to make master work (like I did).

tomjn commented 5 years ago

@coderkevin PRs for the release post and the merge to master are ready and awaiting final approval, it shouldn't be more than an hour or two

tomjn commented 5 years ago

And it's been released 🚀 master should be ok to use now

@cagross the changes to the database restore script got merged into the release

I expect to do a 3.1 release within the next 2 weeks to polish and cleanup things that are still outstanding, thanks to everybody for being patient and helping out, it's been super helpful :)

janw-me commented 5 years ago

Was this a major release on a friday afternoon :o Bold move

cagross commented 5 years ago

@tomjn

the changes to the database restore script got merged into the release

But I can't test those changes because I'm still unable to vagrant up, due to the VirtualBox issue described above.

unfulvio commented 5 years ago

Since switching to 3.0.0 I've bumped into some issues.

I'm still trying to complete a full first vagrant up and I can't finish the provisioning process for a reason or another.

While I tackle the rest of the issues, I could mention that, for example, the network_detection() check appears to be inconsistent. Sometimes it complains that couldn't ping launchpad.net and fails. However, I'm always able to manually ping after ssh'ing into the box. Since the problem has been intermittent, I couldn't pin down the cause and I've decided to edit provision-network-functions.sh to always return ping_result="Connected", so it always passes.

Once I do so, the provisioning script seems having no issue related to connectivity, so there must be something wrong in the way the utility function determines whether an internet connection is present. This happened on a newly cloned repository, without any customization present. I'm running Fedora 30 and VirtualBox 6.

tomjn commented 5 years ago

@unfulvio we ping launchpad as that's where apt packages get installed from, if you've a better or more specific domain to ping I'm open to suggestions

unfulvio commented 5 years ago

I'm not sure. I'd like to be able to give an educated answer on why wget failed on me, for no apparent reason, while the script was running. I would say try ping instead, but that might not be equally reliable. So, for now I'd merely suggest to present the user running the script a question in case they want to continue despite the connectivity check failure; perhaps also mention in the message the possibility thing may actually work, and invite them to try pinging launchpad while ssh'ing the machine.

kraftbj commented 5 years ago

A few random things:

  1. I can't the automated hosts management. It'll fail for permissions, and fail if I try to sudo it for kicks. I have to manually copy/paste the lines from the console into the HOSTS file.

When running phpunit after provisioning the trunk install, I have to manually setup the wp-tests-config.php file, guessing that it uses the same creds as production and guessing at the db name based on checking myphpadmin.

With the trunk install, I wanted to grunt build, but grunt wasn't installed. Tried npm install grunt and sudo apt-get install grunt but get errors instead.

It seems like building and unit testing should be ready to go out of the box; not sure if I did something wrong or if something got missed.

tomjn commented 5 years ago

@kraftbj thanks for testing, do you have any provisioner logs? It's difficult to debug without them, I'll try but I can only speculate on most things. Was this a fresh install? Is this on MacOS or on Linux? And did you use the custom-site-template-develop for trunk? Or the deprecated vvv-wordpress-develop?

I can't the automated hosts management. It'll fail for permissions, and fail if I try to sudo it for kicks. I have to manually copy/paste the lines from the console into the HOSTS file.

I don't know the settings on your hosts file, but the hostsupdater plugin should automatically ask you for your password so it can get the priviledges needed to modify the hosts file. Unless your in a position were sudo is unable to modify the hosts file, or you have software actively interfering ( antivirus, or even programs like gas mask which undo any changes it makes shortly after they're made ), then that's not something I can help with from a VVV point of view as that plugins a separate project

Also, never run vagrant commands with sudo

With the trunk install, I wanted to grunt build, but grunt wasn't installed. Tried npm install grunt and sudo apt-get install grunt but get errors instead.

If I SSH in and run which grunt I get /usr/bin/grunt. It's difficult to debug further without a provisioner log

kraftbj commented 5 years ago

@tomjn I'll try to delete everything fresh again and get a log. I ended up deleting everything struggling to get VVV 3 working as an upgrade to 2. Vagrant kept trying to use a cached trusty box vs what VVV 3 wanted to use.

Using MacOS. Getting the trunk development by switching the skip provisioning flag in vvv-custom.yml this section:

  # It uses the built WP to serve the site
  wordpress-trunk:
    skip_provisioning: false # provisioning this one takes longer, so it's disabled by default
    description: "An svn based WP Core trunk dev setup, useful for contributor days, Trac tickets, patches"
    repo: https://github.com/Varying-Vagrant-Vagrants/custom-site-template-develop.git
    hosts:
      - trunk.wordpress.test

Looking at custom-site-template-devlop.git, I don't see anything there that would generate the tests config file. It exists in the main VVV repo , but didn't see how it was supposed to be created. My first read through this codebase, so I may be missing something.

Anyhow, I'm nuking VVV and trying fresh to see what I can duplicate.

kraftbj commented 5 years ago

from provisioning the wordpress-trunk by setting true to false in the the vvv-custom.yml block:

    default: grunt-cli: The grunt command line interface (v1.3.2)
    default: Fatal error: Unable to find local grunt.
    default: If you're seeing this message, grunt hasn't been installed locally to
    default: your project. For more information about installing and configuring grunt,
    default: please see the Getting Started guide:
    default:
    default: https://gruntjs.com/getting-started
    default: Grunt initialized.

Getting the same if ssh'ing in and vagrant@vvv:/srv/www/wordpress-trunk/public_html$ grunt. We need to npm install before running Grunt though it looks like it should. I'll send you my provisioning log.

But, I'm getting errors randomly. I'm guessing hitting some kind of issue with too much I/O with the VM?

vagrant@vvv:/srv/www/wordpress-trunk/public_html$ npm install
npm ERR! path /srv/www/wordpress-trunk/public_html/node_modules/anymatch/node_modules/kind-of/package.json.1331030607
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/srv/www/wordpress-trunk/public_html/node_modules/anymatch/node_modules/kind-of/package.json.1331030607'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/vagrant/.npm/_logs/2019-05-21T16_34_28_360Z-debug.log
vagrant@vvv:/srv/www/wordpress-trunk/public_html$ grunt
Loading "Gruntfile.js" tasks...ERROR
>> Error: Cannot find module 'webpack/lib/RequestShortener'
Warning: Task "default" not found. Use --force to continue.

Aborted due to warnings.
vagrant@vvv:/srv/www/wordpress-trunk/public_html$ npm install
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.7 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.7: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! path /srv/www/wordpress-trunk/public_html/node_modules/anymatch/node_modules/extglob/node_modules/define-property/package.json.475390675
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/srv/www/wordpress-trunk/public_html/node_modules/anymatch/node_modules/extglob/node_modules/define-property/package.json.475390675'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/vagrant/.npm/_logs/2019-05-21T16_35_16_094Z-debug.log
vagrant@vvv:/srv/www/wordpress-trunk/public_html$ rm -rf node_modules/
vagrant@vvv:/srv/www/wordpress-trunk/public_html$ npm install
npm ERR! path /srv/www/wordpress-trunk/public_html/node_modules/applause/node_modules/esprima/package.json.3053898847
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall open
npm ERR! enoent ENOENT: no such file or directory, open '/srv/www/wordpress-trunk/public_html/node_modules/applause/node_modules/esprima/package.json.3053898847'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

On the wp-tests-config.php issue, there still isn't one in that install. There's the expected wp-tests-config-sample.php, but nothing converted it to something usable.

aaronware commented 5 years ago

@cagross did you take a look at your user ~/VirtualBox VMs and see if there was an older directory that still existed even after destroy? That was my issue and I had to rename the folder (you could also delete it). After I manually changed the folder, I no longer received the error and everything spun up correctly the next time I ran vagrant up --provision

cagross commented 5 years ago

@aaronware Thanks for that. Unfortunately, due to the continuing issues I was having, I had to switch to a different local development solution (Local by Flywheel). It seems to do what I need, so I'm going to have to stick with it for the foreseeable future. If I ever return to VVV, I'll keep your info in-mind. Thanks.

tomjn commented 5 years ago

Hello!

To all the people who had to disable the shared DB folder mount, I've a task for you.

Can you SSH into the VM with vagrant ssh and run id mysql then tell me the result? In particular I'm interested in the ID numbers for the mysql user and group for comparison

ghost commented 5 years ago

I'm on Elementary OS ( Linux ) with VirtualBox 6.0.8 and Vagrant 2.2.4

vagrant@vvv:~$ id mysql
uid=112(mysql) gid=115(mysql) groups=115(mysql),999(vboxsf)
tomjn commented 5 years ago

I'm going to close this out so we can focus on specific issues on their own rather than a general singule issue with everything piled in

lock[bot] commented 4 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.