search

aws / amazon-ssm-agent

An agent to enable remote management of your EC2 instances, on-premises servers, or virtual machines (VMs).
https://aws.amazon.com/systems-manager/
Apache License 2.0
1.05k stars 324 forks source link

aws:downloadContent fails to clone git repositories when a specific branch is provided via getOptions #424

Open jlgoolsbee opened 2 years ago

jlgoolsbee commented 2 years ago

I'm trying to use the aws:downloadContent plugin to clone/checkout a specific branch of a git repository, but when configured to do so, the action always fails with an error. This occurs with any repo/branch combination I have tested thus far. Below I've listed a random sampling of repositories, hosts (just to rule out any one git host), and branches; none are the repo I'm attempting to clone, which is private, but all of which exhibit the same error and return successfully if the getOptions key/value parameter is removed:

Based on my testing, this issue is limited to the branch parameter for getOptions; specifying a commitID works (but is not a workaround for this issue).

Docs: https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-plugins.html#aws-downloadContent

Steps to reproduce:

  1. Create an SSM document with the aws:downloadContent plugin
  2. Set the SourceType parameter to "Git"
  3. Set the SourceInfo parameter to any git repo with the getOptions parameter and a branch name

Expected behavior:

The agent clones (or checks out) the specified branch.

Actual behavior:

The action fails and throws the error Cannot checkout: reference not found.

SSM Agent version: 3.1.821.0 Target OS: Amazon Linux 2

Example SSM document:

---
schemaVersion: "2.2"
description: Download the configured git repo

parameters:
  SourceInfo:
    default: {}
    displayType: "textarea"
    description: "Specify the information required to access the git repository."
    type: "StringMap"

mainSteps:
  - action: aws:downloadContent
    name: downloadContent
    inputs:
      SourceInfo: "{{ SourceInfo }}"
      SourceType: "Git"
    precondition:
      StringEquals:
        - platformType
        - Linux
  - action: aws:runShellScript
    name: runShellScript
    inputs:
      timeoutSeconds: "5"
      runCommand:
        - "#!/usr/bin/env bash"
        - "git status"
    precondition:
      StringEquals:
        - platformType
        - Linux
jlgoolsbee commented 2 years ago

I've opened AWS Support Case ID 9557031041 with links to the SSM doc I've been using to test this behavior along with execution IDs which exhibit the behaviors as-described above.

jlgoolsbee commented 2 years ago

After some back-and-forth with AWS Support, in their view this is a documentation issue, in that to use a branch other than the default branch, the current documentation which outlines the format as branch:branch_name is incorrect. The format should instead be branch:branch_reference, where branch_reference is either:

  1. a local reference in the format refs/heads/branch_name
  2. a remote reference in the format refs/remotes/origin/branch_name

I think as a matter of practicality only the second of those is useful, as it seems the only local reference that exists when a repo is cloned with the aws:downloadContent plugin is the reference to the default branch, which the plugin downloads without error without the getOptions parameter (even when the default branch isn't "master" as the documentation also points out, also incorrectly).

So, below I've reused the examples above to illustrate values for getOptions that work:

{ "repository": "https://github.com/aws/amazon-ssm-agent.git", "getOptions": "branch:refs/heads/mainline" }
{ "repository": "https://gitlab.com/pycqa/flake8.git", "getOptions": "branch:refs/remotes/origin/pre-commit-ci-update-config" }
{ "repository": "https://bitbucket.org/atlassianlabs/atlascode.git", "getOptions": "branch:refs/remotes/origin/VSCODE-897-testing" }

I'm leaving this issue open at least until the documentation has been updated; I think a better user experience here would be for the plugin to function as-currently-documented, but I have no indication/expectation that AWS will take that approach.

andyeff commented 2 years ago

Thanks Lee, been struggling with the sourceType of Github and trying to use non-default branches. Wasn't working with the updated branch_reference format either after I found your issue.

I've since switched to sourceType Git, and have got both HTTPS and SSH-based downloadContent tasks working from any branch, so this has cured a long-running headache :)

mokeeffe-digistorm commented 1 year ago

Thank you @jlgoolsbee for your detailed comment. I have just experienced the same issue and had to put the refs/remotes/origin/ prefix on my branch name in order to get this working.