v1.15 Test Plan

[rfay]

It's testing week here in the DDEV-Local factory. We'd love community help, especially with validating the new nginx configurations for your favorite CMSs, but also with every other item that is interesting or relevant to you. Please use ddev v1.15.0-rc3 for your testing, it's available in homebrew (macOS, Linux, WSL2) and Chocolatey, and of course you can install other ways as shown in the release notes.

If you find a bug that you're pretty sure is a bug, then just open an issue for it.

To give feedback or ask questions, please just reply on this issue or discuss in slack or another other support channel.

1. MariaDB 10.5:
   • Install Drupal9 using it, then take a tour
   • Verify that ddev snapshot and ddev restore-snapshot work
   • Verify that export-db and import-db do the right thing and result in a working site.
2. DDEV-Live integration:
   • Checkout a D8 site
   • ddev auth ddev-live with a valid token for the site
   • ddev config ddev-live to set up the site
   • ddev pull to update database and files
   • Verify site correctness
   • Use ddev live <command>, for example, ddev live list sites
3. BASH_ENV and addition of vendor/bin to PATH: Verify that ddev exec 'echo $PATH' contains /var/www/html/vendor/bin
4. zsh, oh-my-zsh and fish autocompletion:
   • Install ddev via homebrew
   • Run zsh and verify that ddev autocompletion works
   • Run fish and verify that ddev autocompletion works
   • Install manually to oh-my-zsh and verify that it works.
   • Test Powershell completion using the Powershell download and the technique in https://superuser.com/questions/886951/run-powershell-script-when-you-open-powershell
5. Laravel support
   • Use manual testing approach in #2249 to create a new laravel site and verify it.
6. DRUSH_OPTIONS_URL instead of drush/drush.yml
   • Create a D8 site from scratch using the quickstart
   • Verify that drush/drush.yml does not exist
   • Verify that ddev exec drush uli works, with correct URL
7. ddev launch problems with specified URL
   • See #2304 for the problem
   • Change the router_http_port and router_https_port to alternate ports
   • Use ddev launch, ddev launch -p and ddev launch -m - they should all work correctly
8. Internet timeout improvements
   • Verify that it's possible to increase the timeout in global.yaml
   • Test in slow environment (vm with network conditioning)
9. WSL2 documentation
   • Run through the WSL2 install documentation from scratch on a clean machine.
10. New apache and nginx configs, and new way of handling them.
    • Test with a real site on each supported CMS (including php), both apache and nginx
    • Test second docroot for apache and nginx.
11. Global custom commands
    • Add a global custom command and make sure it works correctly.
    • Add a project-level custom command that "shadows" the global one and verify that the project-level command overrides the global.
    • Make sure that CRLF in both global and project-level custom commands are detected and notified (and the command is not installed)
12. Global homeadditions
    • Add contents to ~/.ddev/homeadditions
    • Verify that they get added to the web container home directory on ddev start
    • Verify that any project-level homeadditions of the same name overwrite any global items.
13. Export-db handling when project stopped, in wrong directory, etc. Make sure stderr goes the right place.
    • ddev export-db --gzip=false >junk.txt should show an error when not in a project directory, and junk.txt should be empty.
    • ddev export-db --gzip=false >junk.sql in project directory should start the project and then export the file.
14. Project renaming in same directory
    • Try to rename an existing project with ddev config --project-name=<newname> and then ddev start it. It should give an error and the path to solve the problem.
15. Hook output streamed as it occurs
    • Add a hook that does a few things slowly - use the example in #2335
    • ddev start and watch as the output happens
    • Verify that if input is requested during the hook execution it can be done, for example

      hooks:
      post-start:
      - exec: "echo hi there && for item in 1 2 3 4 5; do echo $item: $(date); sleep 1; done"
      - exec: "echo 'OK? Y/n' && read x"

16. Verify that web image behavior is compatible with v1.14 (after rebuild of image with ddev-images)
    • Packages installed (dpkg -l, get list, compare)
    • Configuration files (docker cp /etc/php from each container and compare the results)
17. Verify that instrumentation/telemetry behaves correctly, and that people are asked to opt in on a new version.
18. Documentation review

[rfay]

My results (updated as I go along). Testing mostly on Windows 10 Home with WSL2 and Ubuntu 20.04, Docker 2.3.0.3.

1. ✅ Mariadb 10.5: Worked fine in all cases, from install/export/import/snapshot/etc

2. ✔️ DDEV-Live integration: (problems fixed in #2348): Tried this with the site rfay/d8c1. Everything went fine until

   • ddev pull, where I got "Pull failed: unable to pull ddev-live database backup (output=`failed waiting for ddev-live pull db: failed to open file for writing: open d8c1-h67nj.sql.gz: permission denied". This turned out to be a WSL2 (only) docker bug, solved in #2364 .

   • 2348: ddev pull --skip-db got me farther, but the output said as below, implying that the files were imported, but nothing was in sites/default/files

     Files downloaded to: /home/rfay/workspace/d8c1/.ddev/.ddevlive-downloads/files Importing files... Pull succeeded."

3. ✅ BASH_ENV: Path correctly shows vendor/bin in the path.

4. ✅ zsh, fish, PowerShell autocompletions: ✅ Both zsh and fish show autocompletions very nicely after brew install drud/ddev-edge/ddev on macOS. ✅ zsh and fish are fine on Ubuntu (WSL2) if brew install ddev and brew install zsh-completions bash-completions was done. But the more normal thing on Ubuntu would be to be using apt-get install bash-completion, which requires manual placement of the file inside /etc/bash_completion.d and ✔️ There's quite a bit of work to be done on the docs for completion. I plan to add significantly to docs before the release.(Improved in #2360

5. ✅ Laravel: I used the two different items from #2249 and they worked perfectly.

6. ✅ DRUSH_OPTIONS_URI: This worked perfectly on Drupal8.9, and it's good because if you do the ddev composer require drush/drush as in the quickstart, it creates an empty drush.yml which would prevent ddev from creating it anyway.

7. ✅ ddev launch with changed port: This worked correctly; after changing port to 8443, ddev launch -p and ddev launch -m worked fine.

8. ✅ Internet timeout improvements: This worked fine; I tested it this time with a Windows 10 Home parallels VM running WSL. The only surprising thing compared to recent testing with an Ubuntu 20.04 VM was that I had to raise the timeout to 3000 ms to accomodate a 1000ms slowdown in Parallels. With Ubuntu 20.04, it worked fine with just 250ms above the slowdown.

9. ✔️ WSL2 documentation: It was super valuable to work this through on a clean machine. There are a couple of things that just didn't work, but most worked fine. I have notes on what to add in my docs pass. Fixed in #2360. It will improve over time.

10. ✅ New apache and nginx configurations: Most actually worked great and it was amazing to work through all the quickstarts in a Windows 10 Home VM with WSL2. It was FAST, and they worked great. BUT ✔️ (fixed) Both magento and magento2 didn't get a custom config, they got the default config (this turned out to be that their comment was not correct)

11. ✅ global custom commands: This worked fine; I tested shadowed commands and commands with CRLF in them, and they all behaved.

12. ✅ Global homeadditions: Worked fine, I tested shadowed files, global, etc.

13. ✔️ (fixed in #2356) export-db handling when project stopped, general stderr output: Many things worked well here. In a project, it did fine. It also did fine with a util.warning emitted (went to stderr). Unfortunately, the simple case of ddev export-db >junk.sql.gz results in all the docker-compose output going into the file and corrupting it. The docker-compose output will need to be redirected. Also ddev output like "Creating ddev-router" and the rest.

    Pushed mkcert rootca certs to ddev-global-cache
    Creating volume "ddev-junk2_nfsmount" with local driver
    Building db
    Building web
    Creating ddev-junk2-db ...
    ESC[1AESC[2K^MCreating ddev-junk2-db ... ESC[32mdoneESC[0m^MESC[1BCreating ddev-junk2-dba ...
    Creating ddev-junk2-web ...
    ESC[2AESC[2K^MCreating ddev-junk2-dba ... ESC[32mdoneESC[0m^MESC[2BESC[1AESC[2K^MCreating ddev-junk2-web ... ESC[32mdoneESC[0m^MESC[1B
    Creating ddev-router ...
    ESC[1AESC[2K^MCreating ddev-router ... ESC[32mdoneESC[0m^MESC[1B
    Ensuring write permissions for junk2
    Ensuring write permissions for junk2
    Existing settings.php file includes settings.ddev.php

14. ✅ Project renaming in same directory: Worked great. I'm sure there's still a way around it. I hope this checking doesn't get in anybody's way.

15. ✅ Streaming hooks: Worked great, and was even able to interact with the hook output.

16. Re-review compatibility of new web image with previous behavior.

17. ✔️ global_config last_started_version is not getting updated on new version, and people aren't being asked to opt in? That could affect the instrumentation. This was a result of using prerelease values like v1.15-rc3 instead of v1.15.0-rc3. It was fixed by using the new numbering. I don't like the semver package we're using.

18. ✅ Documentation review. #2360

[mglaman]

I just tried this, and ddev config ddev-live came up as an invalid argument.

[rfay]

What version of ddev were you using @mglaman ? Was this the first command you had ever issued in that project? I twould still think it would work. Had you done the required ddev auth ddev-live?

But anyway, if you don't have edge ddev, this won't work, as v1.15 is unreleased. To test this you would be using https://github.com/drud/ddev/releases/tag/v1.15.0-rc3 or later.

[mglaman]

@rfay I downloaded https://github.com/drud/ddev/releases/download/v1.15.0-rc3/ddev_macos.v1.15.0-rc3.tar.gz. I was able to use ddev auth ddev-live and provide my token. But ddev-live came up as an invalid argument for ddev config.

[rfay]

I'm unable to recreate your result. If you could do a ddev --version it would help me, and show the entire sequence of commands.

$ ddev --version
ddev version v1.15.0-rc3
rfay@rfay-mbp-2017:~/workspace/ddltest$ ddev config ddev-live
You are reconfiguring the project at /Users/rfay/workspace/ddltest.
The existing configuration will be updated and replaced.
Project name (ddltest):

[rfay]

I watched your stream @mglaman . You downloaded the v1.15 version, but never put it into your path. For a little while (including the auth) you were using an explicit path to the new one. But then you copied a command (ddev config ddev-live) that did not include the explicit path.

[mglaman]

🤦‍♂️I did the first time to auth, and then probably forgot on the second time. Darn. Thanks for the catch. Writing the summary blog

[rfay]

I think all the concerns in https://github.com/drud/ddev/issues/2344#issuecomment-651240678 have been taken care of and v1.15.0 has been released.