Push results in DB reset

[seanr]

I've set up the integration, which seems to work, but when I push new code, the db-update fails and my site redirects to the installer. Something is apparently wiping out the database in dev on each push. :scream:

This shows up as a failed step in my build process:

#!/bin/bash -eo pipefail
terminus -n drush "$TERMINUS_SITE.$TERMINUS_ENV" -- updatedb -y
Warning: Permanently added '[appserver.dev.58920aaf-62bf-42b5-b885-70539047325c.drush.in]:2222,[35.193.88.141]:2222' (RSA) to the list of known hosts.
In BootstrapHook.php line 32:
  Bootstrap failed. Run your command with -vvv for more information.  
updatedb [--cache-clear [CACHE-CLEAR]] [--entity-updates] [--post-updates [POST-UPDATES]] [--no-cache-clear] [--no-post-updates] [-h|--help] [-q|--quiet] [-v|vv|vvv|--verbose] [-V|--version] [--ansi] [--no-ansi] [-n|--no-interaction] [-d|--debug] [-y|--yes] [--no] [--remote-host REMOTE-HOST] [--remote-user REMOTE-USER] [-r|--root ROOT] [-l|--uri URI] [--simulate] [--pipe] [-D|--define DEFINE] [--druplicon] [--xh-link XH-LINK] [--notify] [--] <command>
 [notice] Command: ************.dev -- drush updatedb [Exit: 1]
 [error]   
Exited with code exit status 1
CircleCI received exit code 1

The bootstrap of course fails because the DB has already been borked at this point, but I can't see where that'd be happening. After restoring the DB, the same db-update run from my local terminus works fine.

[stevector]

@seanr can you share more of your log? The db should be copied from Live to Dev as part of the CI process.

[seanr]

@seanr can you share more of your log? The db should be copied from Live to Dev as part of the CI process.

LOL, so that's exactly the problem, then. Live doesn't exist yet as this is a site currently in development, @stevector. Unfortunately, I don't know how to get a copy of the entire dump out of CircleCI - please tell me I don't have to copy each build step one at a time?

[stevector]

The workflow in this repo presumes the existence of a live environment. From the README

Next, deploy to the Pantheon Test and Live environments. When pull requests are created on your GitHub repo they will generate Multidev environments on Pantheon that clone the database and files from the Live environment. Be sure to replace machine-name-for-new-site with the machine name you chose above.

The orb has a clone_content param. Setting that to false might be the workaround you need. https://github.com/pantheon-systems/circleci-orb#parameters

[seanr]

@stevector what's the best setup to use where there is content in dev related to new functionality that is not yet in live? This functionality makes sense to me for multidevs (pulling from a specified environment), but deploying directly to dev (pushing commits on master) shouldn't be doing anything to the dev DB other than running the updates and clearing caches, IMO. On a larger project, I'd be using feature branches, of course, but this is a small one early on and I'm the only dev on it. ;)

[stevector]

Yeah the baseline Pantheon workflow assumes that code changes and content changes are flowing in opposite directions.

That assumption is can be actively unhelpful in very early project states if you're working on content and code/config simultaneously and you don't want the arrows of changes flowing in opposite directions. If the workflow you're doing right now is one where the meaningful changes to both code and content need to happen in the same place, I think you might be better off treating a local environment as the place changes to both happen and GitHub and a Pantheon DB server act as recipients of those changes. Or just setting clone_content param to false might be a good enough middle ground.

But maybe I should first ask, for the project you're working on now, do you expect to get soon get to a point where code and content changes are flowing in opposite directions? Because if this project isn't aiming for that workflow, then the project needs are fighting against the workflow these tools were designed for.

[seanr]

It will, but it just got started. Thanks for the help.

To make this issue actionable, I would suggest a check that the live environment (or whichever has been configured to clone from) actually exist before initiating the clone.

Furthermore, the readme only mentions cloning to multidev, not to the dev environment. Really, we should not be wiping dev ever, instead letting the dev team (assuming a bigger project than mine) chose when to do that.

A very common workflow on big projects I've been on have the bottom arrows on that workflow graph basically stop at test, with further pulls down to dev being arbitrary and explicitly manual depending on the project status. When the site launches, automated Live->Test->Dev clones like you seem to describe would be fine, but now big project 2 begins and we're creating a bunch of test content on dev to show the client functionality of new features we're building. Maybe we're adding a complex store (this actually describes exactly my last project, using Drupal Commerce and D9, but on Acquia). There's a ton of stuff being done on dev that will never see the light of day because it'll be replaced by real content when such is read, but must be there both to test integrating functionality and to show sprint updates to the client.

That scenario would be broken by this tool as I've experienced it even though live would exist (and would in fact be live) because now you're blowing away our dev content with each push/merge, apparently. Because of this, I actually think clone_content should be off by default for pushes that land on dev, but multidevs should still clone content from the configured environment when created (but probably only when created, not for subsequent pushes).

So that leaves one remaining question: with clone_content set to false, what state will new multidevs be created in? Can we say multidevs clone content from dev (or live or test, as configured) when spun up so as not to be completely empty sitting at the install page, but no other actions clone content around? That would honestly be ideal.

[stevector]

Thanks for the input @seanr! I'm transferring this issue here to the circleci-orb repo. And arguably this issue belongs in https://github.com/pantheon-systems/terminus-build-tools-plugin cc @sugaroverflow