Pack registration does not fetch local changes of a pack that has a repository (with .git folder inside)

[Thecave3]

SUMMARY

When you try to remove and install a pack from a local copy of a git repository st2 will fetch the remote version of the copy and not the local version. I have also tried to edit the version number from local copy in 'pack.yaml', but nothing has changed. It seems that everytime that i need to reload a pack I have to make a forced commit.

STACKSTORM VERSION

docker version with python3 added: st2 3.1.0, on Python 2.7.6

OS, environment, install method

git clone of st2-docker on Ubuntu + installation of python 3 by using this script:

if [ ! -f /usr/bin/python3.6 ]; then

    # Install PPA repo and python3.6
    apt-get update && apt-get install -y software-properties-common
    add-apt-repository -y ppa:jonathonf/python-3.6
    apt-get update && apt-get install -y python3.6 nano # comment this in production

    # Update bindings for python3
    update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.4 1
    update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 2

fi

Steps to reproduce the problem

1. Take a pack with .git folder inside
2. Run st2 pack remove <name of pack > && st2 pack install file:///<path of local copy> --python3

Expected Results

I expect that changes will be installed on the pack and I expect that st2 will fetch them from the local copy of repository.

Actual Results

Apparently it seems that st2 takes the remote copy everytime. It may be usefull to know that i was working not in master branch on my repository when i encountered this bug. I order to fix that behaviour for myself I had to make a commit everytime i make changes in repository.

[blag]

Are you trying to install from the local copy in /opt/stackstorm/packs?

st2 pack install file:///opt/stackstorm/packs/...

If so, don't. Leave everything in /opt/stackstorm/packs/ alone, that should all be managed only by StackStorm itself.

ST2 copies local packs into /opt/stackstorm/packs as part of the installation process. So if you need to keep a pack on the ST2 filesystem, you'll need to keep it outside of /opt/stackstorm/packs, but still accessible to the stanley user.

Finally, regarding being forced to make a git commit, please see the warning in this section of our pack documentation. That same section also has information for how to install a pack from a specific version, branch, tag, or commit.

[arm4b]

st2 will fetch the remote version of the copy and not the local version

In your case, from where st2 tried to fetch content of the pack? git remote?

[Thecave3]

Are you trying to install from the local copy in /opt/stackstorm/packs?

No, I am trying to install from an external volume linked inside docker which is located in the / folder of the container. To be more clear this is the situation inside the docker container

.
├── bin
├── boot
├── dev
├── dumb-init
├── etc
├── home
├── install.sh
├── lib
├── lib64
├── media
├── mnt
├── pack_updated  # this has a git folder inside
├── opt
├── proc
├── root
├── run
├── sbin
├── srv
├── st2-docker
├── st2-manifest.txt
├── sys
├── tmp
├── usr
└── var

What i am doing is st2 pack install file:///pack_updated

In your case, from where st2 tried to fetch content of the pack? git remote?

Apparently it has taken the latest commit of the remote copy of the repository from a private gitlab server of the organization.

[arm4b]

Thanks for the report.

Definitely sounds like a bug as it shouldn't be fetching remote one, but getting latest commits from the local file:///. This indeed could be improved.

And as @blag noted, st2 pack under the hood heavily relies on pack to be a git repo with committed changes.

[blag]

Are there any remotes configured in the installed pack? I suspect that there's a private gitlab server as a remote.

$ cd /opt/stackstorm/packs/pack_updated
$ git remote -v

If there are, that might be confusing things. Can you remove (or simply comment out) any remotes from that config file and try st2 pack install again?

sudo nano /opt/stackstorm/packs/pack_updated/.git/config

[Thecave3]

Are there any remotes configured in the installed pack? I suspect that there's a private gitlab server as a remote.

Result of the commands:

root@9bcdde787221:/opt/stackstorm/packs/pack_updated# git remote -v
origin  file:///<path of local copy>  (fetch)
origin  file:///<path of local copy>  (push)

I think that this is the expected output, but by inspecting /opt/stackstorm/packs/pack_updated/.git/config I found something strange:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = file:///<path of local copy>
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "juniper"]
        remote = origin
        merge = refs/heads/juniper

I am working on juniper branch but it seems that until the changes are not committed st2 will not fetch them. This is an odd behaviour because it forces me to commit everytime I need to make changes to the code and in this way I simply lose the comfort of working in local.

Can you remove (or simply comment out) any remotes from that config file and try st2 pack install again?

I never touched a git config file before, can you show me how i have to comment the file please? Is the same of git commit messages (#) ?

[winem]

I just reproduced the issue.

To answer the question from @blag Removing / commenting out the remote does not help to fix the issue.