molecule converge -s upgrade fails to clone the git repo

kushaldas commented 5 years ago

Description

The molecule converge -s upgrade command fails to clone the securedrop git repository step.

Steps to Reproduce

molecule converge -s upgrade

Expected Behavior

Command should execute and create the staging molecule instance.

Actual Behavior

$ molecule converge -s upgrade 
--> Validating schema /home/kdas/code/securedrop/molecule/vagrant_packager/molecule.yml.
Validation completed successfully.
--> Validating schema /home/kdas/code/securedrop/molecule/virtualbox-staging-xenial/molecule.yml.
Validation completed successfully.
--> Validating schema /home/kdas/code/securedrop/molecule/qubes-staging/molecule.yml.
Validation completed successfully.
--> Validating schema /home/kdas/code/securedrop/molecule/ansible-config/molecule.yml.
Validation completed successfully.
--> Validating schema /home/kdas/code/securedrop/molecule/upgrade/molecule.yml.
Validation completed successfully.
--> Validating schema /home/kdas/code/securedrop/molecule/libvirt-staging-xenial/molecule.yml.
Validation completed successfully.
--> Validating schema /home/kdas/code/securedrop/molecule/virtualbox-staging/molecule.yml.
Validation completed successfully.
--> Validating schema /home/kdas/code/securedrop/molecule/libvirt-staging/molecule.yml.
Validation completed successfully.
--> Validating schema /home/kdas/code/securedrop/molecule/builder/molecule.yml.
Validation completed successfully.
--> Test matrix

└── upgrade
    ├── create
    ├── dependency
    └── converge

--> Inventory /home/kdas/code/securedrop/molecule/upgrade/../../install_files/ansible-base/group_vars linked to /tmp/molecule/securedrop/upgrade/group_vars
--> Inventory /home/kdas/code/securedrop/molecule/upgrade/../../install_files/ansible-base/host_vars linked to /tmp/molecule/securedrop/upgrade/host_vars
--> Scenario: 'upgrade'
--> Action: 'create'

    PLAY [Bring in SecureDrop production version] **********************************

    TASK [debug] *******************************************************************
    ok: [localhost] => {
        "msg": "Starting scenario from baseline "
    }

    TASK [Ensure molecule local ephemeral dir exists] ******************************
    ok: [localhost]

    TASK [Clone in copy of original SD version] ************************************
    fatal: [localhost]: FAILED! => {"changed": false, "cmd": "/usr/bin/git clone --origin origin --depth 1 --branch '' https://github.com/freedomofpress/securedrop.git /tmp/molecule/securedrop/upgrade/sd-orig", "msg": "Cloning into '/tmp/molecule/securedrop/upgrade/sd-orig'...\nwarning: Could not find remote branch  to clone.\nfatal: Remote branch  not found in upstream origin", "rc": 128, "stderr": "Cloning into '/tmp/molecule/securedrop/upgrade/sd-orig'...\nwarning: Could not find remote branch  to clone.\nfatal: Remote branch  not found in upstream origin\n", "stderr_lines": ["Cloning into '/tmp/molecule/securedrop/upgrade/sd-orig'...", "warning: Could not find remote branch  to clone.", "fatal: Remote branch  not found in upstream origin"], "stdout": "", "stdout_lines": []}

    PLAY RECAP *********************************************************************
    localhost                  : ok=2    changed=0    unreachable=0    failed=1   

ERROR:

Comments

I have tried on VPN (different VPNs) and off the VPN, still same error. This is a fresh clone of the git repository.

conorsch commented 5 years ago

Given that you're referencing the upgrade scenario, check out #3832, since that cleans up both the docs for the upgrade logic, as well as adds a metadata reference to an upgrade box for 0.11.0. Chime in over there, and see if the workflow described there resolves the issue for you.

In short, the error you report is caused by the lack of a "stable version" var for cloning. The error output shows the failing command:

/usr/bin/git clone --origin origin --depth 1 --branch '' https://github.com/freedomofpress/securedrop.git /tmp/molecule/securedrop/upgrade/sd-orig", "msg": "Cloning into '/tmp/molecule/securedrop/upgrade/sd-orig

That --branch '' is the source of the error. The docs in #3832 recommend using a wrapper to ensure that the var is specified; we could strengthen the validation logic a bit to be helpful, e.g.

--- a/molecule/shared/sd_clone.yml
+++ b/molecule/shared/sd_clone.yml
@@ -4,8 +4,19 @@
   hosts: localhost
   gather_facts: false
   tasks:
-    - debug:
-        msg="Starting scenario from baseline {{ ORIG_SECUREDROP_VER }}"
+    - name: Confirm target version for upgrade testing is specified
+      assert:
+        that:
+          - ORIG_SECUREDROP_VER is defined
+          - ORIG_SECUREDROP_VER != ''
+        msg: >-
+          Could not find target SecureDrop version. Make sure declare
+          the SD_UPGRADE_BASE env var, or use the 'make upgrade-start' target
+          to do so automatically.
+
+    - name: Display target version for upgrade testing
+      debug:
+        msg: "Starting scenario from baseline {{ ORIG_SECUREDROP_VER }}"

     - name: Ensure molecule local ephemeral dir exists
       file:

What do you think, @kushaldas ?

kushaldas commented 5 years ago

What do you think, @kushaldas ?

@conorsch this sounds like a good solution.

kushaldas commented 5 years ago

Another common error I am seeing is

Volume for domain is already created. Please run 'vagrant destroy' first.

freedomofpress / securedrop