ruamel.yaml 0.17.32 version buggy for 0.1.22 singularity-hpc release

[audreystott]

When installing -b 0.1.22 on Ubuntu 22.04, shpc appears to install successfully (lines have been omitted):

git clone -b 0.1.22 https://github.com/singularityhub/singularity-hpc.git
cd singularity-hpc
sudo python3 setup.py install
sudo pip3 install .
.
.
.
Searching for ruamel.yaml
Reading https://pypi.org/simple/ruamel.yaml/
Downloading https://files.pythonhosted.org/packages/d9/0e/2a05efa11ea33513fbdf4a2e2576fe94fd8fa5ad226dbb9c660886390974/ruamel.yaml-0.17.32-py3-none-any.whl#sha256=23cd2ed620231677564646b0c6a89d138b6822a0d78656df7abda5879ec4f447
Best match: ruamel.yaml 0.17.32
Processing ruamel.yaml-0.17.32-py3-none-any.whl
Installing ruamel.yaml-0.17.32-py3-none-any.whl to /usr/local/lib/python3.8/dist-packages
Adding ruamel.yaml 0.17.32 to easy-install.pth file

Installed /usr/local/lib/python3.8/dist-packages/ruamel.yaml-0.17.32-py3.8.egg
.
.
.
Successfully installed singularity-hpc-0.1.22

However, it seems the new ruamel version (0.17.32) causes an error when running the shpc config edit command (shpc --help is fine):

shpc config edit

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/shpc/utils/fileio.py", line 17, in <module>
    from ruamel_yaml import YAML
ModuleNotFoundError: No module named 'ruamel_yaml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/shpc", line 8, in <module>
    sys.exit(run_shpc())
  File "/usr/local/lib/python3.8/dist-packages/shpc/client/__init__.py", line 519, in run_shpc
    from .config import main
  File "/usr/local/lib/python3.8/dist-packages/shpc/client/config.py", line 7, in <module>
    import shpc.defaults as defaults
  File "/usr/local/lib/python3.8/dist-packages/shpc/defaults.py", line 7, in <module>
    import shpc.utils as utils
  File "/usr/local/lib/python3.8/dist-packages/shpc/utils/__init__.py", line 1, in <module>
    from .fileio import (
  File "/usr/local/lib/python3.8/dist-packages/shpc/utils/fileio.py", line 19, in <module>
    from ruamel.yaml import YAML
ImportError: cannot import name 'YAML' from 'ruamel.yaml' (/usr/local/lib/python3.8/dist-packages/ruamel.yaml.clib-0.2.7-py3.8-linux-x86_64.egg/ruamel/yaml/__init__.py)

After installing ruamel.yaml 0.17.21 as follows, I can successfully run the shpc config edit command and also other shpc commands:

sudo pip3 install ruamel.yaml==0.17.21

Am I doing something wrong with my initial install commands?

[vsoch]

I don't think so, and I think for the time being we should pin this version in the shpc/version.py. Would you like to submit a PR for that?

[audreystott]

Sure I can do that. ruamel.yaml==0.17.21 for shpc v0.1.22?

[vsoch]

Yep let’s do that for now - thank you! 🙏

[audreystott]

Can you open a new branch for me to do a PR to? I can't do a PR to the 0.1.22 tag.

[vsoch]

the tags are releases, so you wouldn’t do a PR against a release, but rather the current main branch.

To do that you can fork the repository, clone your fork, checkout a branch, do the changes, commit and push, and then open the PR from your fork against main. It’s not typical to allow write access when contributing to a repository. Let me know if you have any questions along the way - happy to help!

[audreystott]

Ok PR created - https://github.com/singularityhub/singularity-hpc/pull/657

[vsoch]

This version will need to be updated (and the bug fixed) as we are broken in conda: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=783260&view=logs&jobId=656edd35-690f-5c53-9ba3-09c10d0bea97&j=656edd35-690f-5c53-9ba3-09c10d0bea97&t=986b1512-c876-5f92-0d81-ba851554a0a3

[vsoch]

@audreystott is this something you might be able to work on? Note that we would want to import ruamel.yaml, and there are examples here: https://yaml.readthedocs.io/en/latest/example.html