S3 Orb broken on CircleCI Provided (non-python) images based on Ubuntu 20.04

[kenperkins]

Orb Version 1.0.16

If you use any CircleCI provided image (non-python) based on 20.04 Ubuntu, for example cimg/go:1.14, this Orb will fail to execute as Ubuntu 20.04 no longer has python in the path, only python3. This does not manifest with cimg/python:3.6 but the orb's aren't much use if I have to do a bunch of os preparation first.

To Reproduce Run this job:

jobs:
  build:
    docker:
      - image: 'cimg/go:1.14'
    steps:
      - checkout
      - run: mkdir bucket && echo "lorem ipsum" > bucket/build_asset.txt
      - aws-s3/sync:
          arguments: |
            --acl public-read \
            --cache-control "max-age=86400"
          aws-access-key-id: AWS_ACCESS_KEY_ID_BLUE
          aws-region: AWS_REGION_BLUE
          aws-secret-access-key: AWS_SECRET_ACCESS_KEY_BLUE
          from: bucket
          overwrite: true
          to: 's3://my-s3-bucket-name/prefix'
      - aws-s3/copy:
          arguments: '--dryrun'
          from: bucket/build_asset.txt
          to: 's3://my-s3-bucket-name'
orbs:
  aws-s3: circleci/aws-s3@1.0.16
version: 2.1

Error Message:

#!/bin/bash -eo pipefail
if [ "false" == "false" ] && which aws > /dev/null; then
  echo "The AWS CLI is already installed. Skipping."
  exit 0
fi

export PIP=$(which pip pip3 | head -1)
if [[ -n $PIP ]]; then
  if which sudo > /dev/null; then
    sudo $PIP install awscli --upgrade
  else
    # This installs the AWS CLI to ~/.local/bin. Make sure that ~/.local/bin is in your $PATH.
    $PIP install awscli --upgrade --user
  fi
elif [[ $(which unzip curl | wc -l) -eq 2 ]]; then
  cd
  curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
  unzip awscli-bundle.zip
  if which sudo > /dev/null; then
    sudo ~/awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
  else
    # This installs the AWS CLI to the default location (~/.local/lib/aws) and create a symbolic link (symlink) at ~/bin/aws. Make sure that ~/bin is in your $PATH.
    awscli-bundle/install -b ~/bin/aws
  fi
  rm -rf awscli-bundle*
  cd -
else
  echo "Unable to install AWS CLI. Please install pip."
  exit 1
fi
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 16.1M  100 16.1M    0     0  77.0M      0 --:--:-- --:--:-- --:--:-- 77.0M
Archive:  awscli-bundle.zip
  inflating: awscli-bundle/install   
  inflating: awscli-bundle/packages/six-1.15.0.tar.gz  
  inflating: awscli-bundle/packages/pyasn1-0.4.8.tar.gz  
  inflating: awscli-bundle/packages/rsa-3.4.2.tar.gz  
  inflating: awscli-bundle/packages/botocore-1.17.62.tar.gz  
  inflating: awscli-bundle/packages/PyYAML-5.3.1.tar.gz  
  inflating: awscli-bundle/packages/colorama-0.4.3.tar.gz  
  inflating: awscli-bundle/packages/futures-3.3.0.tar.gz  
  inflating: awscli-bundle/packages/urllib3-1.25.10.tar.gz  
  inflating: awscli-bundle/packages/virtualenv-16.7.8.tar.gz  
  inflating: awscli-bundle/packages/docutils-0.15.2.tar.gz  
  inflating: awscli-bundle/packages/colorama-0.4.1.tar.gz  
  inflating: awscli-bundle/packages/python-dateutil-2.8.0.tar.gz  
  inflating: awscli-bundle/packages/jmespath-0.10.0.tar.gz  
  inflating: awscli-bundle/packages/s3transfer-0.3.3.tar.gz  
  inflating: awscli-bundle/packages/awscli-1.18.139.tar.gz  
  inflating: awscli-bundle/packages/urllib3-1.25.7.tar.gz  
  inflating: awscli-bundle/packages/PyYAML-5.2.tar.gz  
  inflating: awscli-bundle/packages/setup/setuptools_scm-3.3.3.tar.gz  
  inflating: awscli-bundle/packages/setup/wheel-0.33.6.tar.gz  
/usr/bin/env: ‘python’: No such file or directory

Exited with code exit status 127
CircleCI received exit code 127

Expected behavior Orb would run after installing.

[kenperkins]

Additional context: From the CircleCI Docs:

CircleCI Base Image

image: cimg/base:2020.01

This is a brand new Ubuntu-based image designed to install the very bare minimum. All of the next-generation convenience images that we will be releasing in the coming weeks are based on this image.

When to use it?

If you need a generic image to run on CircleCI, to use with orbs, or to use as a base for your own custom Docker image, this image is for you.

[kenperkins]

I did a quick test with the latest next-gen images (cimg) for node, ruby, rust, openjdk and php: they all are missing python as an interpreter. I expect this orb to not work an any of these as well, but I didn't test the full workflow.

[kenperkins]

Adding this command allows a work around with minimal steps

- run: sudo apt update && sudo apt install python3-venv python-is-python3

[FelicianoTech]

Looking into this.

[nraychaudhuri]

FYI. It failed for me on the sync command even though all the aws creds were in place. I was trying it for the first time. Reverted back to 1.0.11

[FelicianoTech]

My notes:

1. So far this doesn't appear to be an Ubuntu 18.04 vs 20.04 issue. Both images don't have a python binary because both images ship Python v3 by default, which is called with python3.
2. This orb is using an old version of the AWS CLI orb. In particular, the newer AWS CLI orb versions support the AWS CLI v2, which includes its own version of Python. It doesn't rely on any external version at all, which would solve this issue.
3. Alternatively, this issue with python not being available can be solved by checking for apt packages, which would then make this orb Debian/Ubuntu specific.

[kenperkins]

FWIW, We were previously using the 16.04 Machine Executor for much of our work due to the inability to mount volumes in the docker executor, but we've been moving away from that, partly because of the ancientness of 16.04, so that was how we came across this. I do think the orb should be resilient against this kind of problem however, having to push that responsibility to the user makes using the orb less appealing.

[FelicianoTech]

I do think the orb should be resilient against this kind of problem however, having to push that responsibility to the user makes using the orb less appealing.

I completely agree.

@gmemstr I can implement a solution but any opinion on which one is better?

[gmemstr]

I'd be in favour of moving to the newer version of the AWS CLI orb to leverage v2, I'm wary of implementing a fix that only addresses Debian/Ubuntu based images.

[FelicianoTech]

@kenperkins A fix was published in v1.1.1. In the future, this orb will use the AWS CLI v2 which includes its own Python and avoids most of this. For now, the standard installation via Pip occurs, even on a non-Python image.