jsonmerge in GitHub Action and/or python virtual environments

[ravensorb]

I've been hitting a roadblock recently in trying to get a few of my projects setup with virtual environments (using pipenv) and GitHub actions and I think the issue is related to jsonmerge. That said, I cannot figure out why it is related.

Here is an example of a python file:

#!/usr/bin/python3
#########################################################################
#########################################################################
import logging
import argparse
import os, sys, pathlib
import jsonmerge

help('modules')

and here is a build action file

# This is a basic workflow to help you get started with Actions

name: CI

# Controls when the workflow will run
on:
  # Triggers the workflow on push or pull request events but only for the master branch
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

env:
  # Tells pipenv to create virtualenvs in /root rather than $HOME/.local/share.
  # We do this because GitHub modifies the HOME variable between `docker build` and
  # `docker run`
  WORKON_HOME: /home/runner/.local/share/virtualenvs

  LC_ALL: C.UTF-8
  LANG: C.UTF-8

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2

      - name: Set up Python 3.x
        uses: actions/setup-python@v2
        with:
          # Semantic version range syntax or exact version of a Python version
          python-version: '3.x'
          # Optional - x64 or x86 architecture, defaults to x64
          architecture: 'x64'

      # Runs a set of commands using the runners shell
      - name: Updating python pip, wheel, and setup tools
        run: |
          python3 -m pip install --upgrade pip setuptools wheel

      - name: Install pipenv
        run: |
          python3 -m pip install --upgrade pipenv

      - name: Installing packages
        run: |
          #echo "******************** PIPENV Installing Packages (system)"
          #python3 -m pipenv lock
          #python3 -m pipenv install --system
          echo "******************** PIPENV Installing Packages"
          python3 -m pipenv lock
          python3 -m pipenv install

      - name: Running test script
        run: |
          pipenv run jsonmerge-test.py

and here is the error I get

docker exec cmd=[bash --noprofile --norc -e -o pipefail /home/testuser/source/workflow/5] user=
| Traceback (most recent call last):
|   File "jsonmerge-test.py", line 7, in <module>
|     import jsonmerge
| ModuleNotFoundError: No module named 'jsonmerge'
[CI/build]   ❌  Failure - Run test script
Error: exit with `FAILURE`: 1

The weird part is -- if I run a simple script with "help('modules')" -- I get back a lot of all modules and jsonmerge 1.8.0 is in the list. Which has me wondering -- am I doing something wrong?

The other interesting thing to note -- even if I skip using pipenv and just do a pip install inside the action, it still fails to run with the exact same error.

Any chance you have any thoughts/suggestions?

[avian2]

No idea, sorry. As far as I know, jsonmerge should have no problems running from a virtual environment.

[ravensorb]

Weird. Just to double and triple check things I built a clean ubuntu server (v20.04.02) yesterday and installed pipenv. I then ran a simple test with just the python code above with a Pipfile that contains only jsonmerge and it also failed with the same error.

The only way I could get it to work was to do a pipenv install --system which seems to defeat the purpose of the virtual environments. Any chance you could double check and see if you can replicate?

Steps I followed:

• Install Ubuntu Server 20.04.02
• Install pip (sudo apt-get install python3-pip)
• Install pipenv (python3 -m pip install pipenv)
• Run pipenv install (python3 -m pipenv install)
• Run pipenv shell (python3 -m pipenv shell)
• Run test script

Here is a gist with the Pipfile and python script

[ravensorb]

I might have found out how to get it to work. Changing the following

#!/usr/bin/python

to

#!/usr/bin/env python

seems to have gotten everything to work.