Pack Install Doesn't Honor System user, running as root

[bigmstone]

If you attempt to perform a pack install from a git repo it uses root ssh keys.

From docs:

For SSH (URLs starting with git@) auth you have to create a deploy key, and require the system user running the command (stanley or root, depending on your configuration) to have a private key. Deploy keys are more secure than personal access tokens and can be configured on the per-repo basis.

However since we clone from python action and python action is shelled out as root and not system user this action will always use the root keys.

[bigmstone]

Noticed some strange behavior today that made me add the to be verified tag. On scripted install I'm seeing it operate under stanley user but using id_rsa instead of stanley_rsa. So this might be different than originally reported as well as cause for opening up a bug on StackStorm/ansible-st2

[bradym]

I ran into this today attempting to install a pack from a private github repo. I'm posting the below in case it helps to verify/provide more detail about the issue.

The following snippet from the output when it failed indicates that it's attempting to clone it as root:

Cloning into '/root/20b4efc01c6c82ffb96ad617f56f1ddf'...
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.

Looking at the traceback:

Traceback (most recent call last):
  File \"/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py\", line 259, in <module>
    obj.run()
  File \"/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py\", line 155, in run
    output = action.run(**self._parameters)
  File \"/opt/stackstorm/packs/packs/actions/pack_mgmt/download.py\", line 75, in run
    verifyssl=verifyssl, ref=pack_version)
  File \"/opt/stackstorm/packs/packs/actions/pack_mgmt/download.py\", line 102, in _clone_repo
    repo = Repo.clone_from(repo_url, temp_dir)
  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/git/repo/base.py\", line 942, in clone_from
    return cls._clone(git, url, to_path, GitCmdObjectDB, progress, **kwargs)
  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/git/repo/base.py\", line 897, in _clone
    finalize_process(proc, stderr=stderr)
  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/git/util.py\", line 341, in finalize_process
    proc.wait(**kwargs)
  File \"/opt/stackstorm/st2/lib/python2.7/site-packages/git/cmd.py\", line 292, in wait
    raise GitCommandError(self.args, status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git clone -v git@github.com:notTheReal/repo.git /root/20b4efc01c6c82ffb96ad617f56f1ddf

The following lines from /opt/stackstorm/packs/packs/actions/pack_mgmt/download.py seem to show why this is happening -- it's not attempting to run as the system user, it's running as whatever user is used to run the python script:

 71                 try:
 72                     user_home = os.path.expanduser('~')
 73                     abs_local_path = os.path.join(user_home, temp_dir_name)
 74                     self._clone_repo(temp_dir=abs_local_path, repo_url=pack_url,
 75                                      verifyssl=verifyssl, ref=pack_version)

Sorry, I don't know how /opt/stackstorm/packs/packs/actions/pack_mgmt/download.py maps to code in the repo(s) - I used the curl command from https://docs.stackstorm.com/install/index.html to install stackstorm.

Hope this helps.

[djh2020]

I am having a similar issue I think. I am using a private repo on an internal github implementation.

git clone git@mygitserver/myrepo.git works fine when I execute this from the stackstorm server as either root or stanley. (I added a deploy key to the repo for each as I am not sure which is being used)

st2 pack install file://$PWD from within the cloned repo also works fine and the pack is installed

However if I try to do this 2-step process in 1 step it fails. i.e. st2 pack install git@mygitserver/myrepo.git.

The action timesout after 600s with an error Action failed to complete in 600 seconds" and IndexError: list index out of range

I was planning to demo how to install packs using chatops which is why I wanted to do it this way. I suppose I could create my own pack / action to do it in 2 steps though

[djh2020]

Here is the traceback

2017-06-27 19:51:35,397  DEBUG - Using cached token from file "/root/.st2/token-st2admin"
# -------- begin 139842584544336 request ----------
curl -X POST -H  'Connection: keep-alive' -H  'Accept-Encoding: gzip, deflate' -H  'Accept: */*' -H  'User-Agent: python-requests/2.14.2' -H  'content-type: application/json' -H  'X-Auth-Token: 7bb4eec372e5499980da968f51db1a52' -H  'Content-Length: 77' --data-binary '{"force": false, "packs": ["git://mygitserver/myorg/pack_demo.git"]}' http://127.0.0.1:9101/v1/packs/install
# -------- begin 139842584544336 response ----------
{
    "execution_id": "5952b747739f8043af361bef"
}
# -------- end 139842584544336 response ------------

    [  timeout  ] download pack

# -------- begin 139842584544528 request ----------
curl -X GET -H  'Connection: keep-alive' -H  'Accept-Encoding: gzip, deflate' -H  'Accept: */*' -H  'User-Agent: python-requests/2.14.2' -H  'X-Auth-Token: 7bb4eec372e5499980da968f51db1a52' http://127.0.0.1:9101/v1/executions/5952b747739f8043af361bef
# -------- begin 139842584544528 response ----------
{
    "status": "timeout",
    "start_timestamp": "2017-06-27T19:51:35.428307Z",
    "log": [
        {
            "status": "requested",
            "timestamp": "2017-06-27T19:51:35.436000Z"
        },
        {
            "status": "scheduled",
            "timestamp": "2017-06-27T19:51:35.498000Z"
        },
        {
            "status": "running",
            "timestamp": "2017-06-27T19:51:35.556000Z"
        },
        {
            "status": "timeout",
            "timestamp": "2017-06-27T20:01:35.950000Z"
        }
    ],
    "parameters": {
        "packs": [
            "git://mygitserver/myorg/pack_demo.git"
        ]
    },
    "runner": {
        "runner_module": "action_chain_runner",
        "uid": "runner_type:action-chain",
        "enabled": true,
        "name": "action-chain",
        "runner_parameters": {
            "display_published": {
                "default": false,
                "type": "boolean",
                "description": "Intermediate published variables will be stored and displayed."
            },
            "skip_notify": {
                "default": [],
                "type": "array",
                "description": "List of tasks to skip notifications for."
            }
        },
        "id": "594ab80d739f8032042de1ca",
        "description": "A runner for launching linear action chains."
    },
    "id": "5952b747739f8043af361bef",
    "elapsed_seconds": 600.483209,
    "web_url": "https://myserver.mycompany.com/#/history/5952b747739f8043af361bef/general",
    "result": {
        "tasks": [
            {
                "name": "download pack",
                "workflow": null,
                "created_at": "2017-06-27T19:51:35.620432+00:00",
                "updated_at": "2017-06-27T20:01:35.881461+00:00",
                "state": "timeout",
                "result": {
                    "result": "None",
                    "error": "Action failed to complete in 600 seconds",
                    "exit_code": -9,
                    "stderr": "",
                    "stdout": ""
                },
                "id": "download pack",
                "execution_id": "5952b747739f8025c0b8f37c"
            }
        ]
    },
    "context": {
        "rbac": {
            "user": "stanley",
            "roles": [
                "admin"
            ]
        },
        "user": "stanley"
    },
    "action": {
        "runner_type": "action-chain",
        "name": "install",
        "parameters": {
            "register": {
                "default": "all",
                "type": "string",
                "description": "Possible options are all, sensors, actions, rules, aliases, runners, triggers, rule_types, policiy_types, policies, configs."
            },
            "force": {
                "default": false,
                "required": false,
                "type": "boolean",
                "description": "Set to True to force install the pack and skip StackStorm version compatibility check and also delete and ignore lock file if one exists."
            },
            "env": {
                "required": false,
                "type": "object",
                "description": "Optional environment variables."
            },
            "packs": {
                "items": {
                    "type": "string"
                },
                "required": true,
                "type": "array",
                "description": "Name of the pack in Exchange or a git repo URL."
            }
        },
        "tags": [],
        "enabled": true,
        "entry_point": "workflows/install.yaml",
        "notify": {},
        "uid": "action:packs:install",
        "pack": "packs",
        "ref": "packs.install",
        "id": "594ab80e739f8032042de202",
        "description": "Installs or upgrades a pack into local content repository, either by git URL or a short name matching an index entry. Will download pack, load the actions, sensors and rules from the pack. Note that install requires reboot of some st2 services."
    },
    "liveaction": {
        "runner_info": {
            "hostname": "myserver.mycompany.com",
            "pid": 9664
        },
        "parameters": {
            "packs": [
                "git://mygitserver/myorg/pack_demo.git"
            ]
        },
        "action_is_workflow": true,
        "callback": {},
        "action": "packs.install",
        "id": "5952b747739f8043af361bee"
    },
    "children": [
        "5952b747739f8025c0b8f37c"
    ],
    "end_timestamp": "2017-06-27T20:01:35.911516Z"
}
# -------- end 139842584544528 response ------------

ERROR: list index out of range

CLI settings:
----------------
Config file path: /root/.st2/config
Client settings:
----------------
ST2_BASE_URL: http://127.0.0.1
ST2_AUTH_URL: http://127.0.0.1:9100
ST2_API_URL: http://127.0.0.1:9101/v1
ST2_AUTH_TOKEN: 7bb4eec372e5499980da968f51db1a52

Proxy settings:
---------------
HTTP_PROXY: 
HTTPS_PROXY: 

Traceback (most recent call last):
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2client/shell.py", line 298, in run
    args.func(args)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2client/commands/resource.py", line 44, in decorate
    return func(*args, **kwargs)
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2client/commands/pack.py", line 200, in run_and_print
    packs = instance.result['tasks'][1]['result']['result']
IndexError: list index out of range

[keatsfonam]

I am seeing the same issue

[cognifloyd]

I wish the git clone would use the system user (st2) instead of root. I don't like using root to git clone anything. I'm on CentOS 7

[mickmcgrath13]

Adding a specific identity file to the global ssh_config for our restricted domain works, too.

Add

Host github.restricteddomain.com
        IdentityFile /path/to/id_rsa

to /etc/ssh/ssh_config

(rhel7)

[cognifloyd]

The reason pack install runs as root, is because all actions run as root, because actionrunner runs as root.

So, actionrunner runs as root because of https://github.com/StackStorm/st2-packages/pull/195 and https://github.com/StackStorm/st2-packages/pull/205

I am specifically not happy about adding a user that has passwordless sudo privileges as part of packaging.

I think you are right, agreed. Thus I will revert actionrunner to run as root and corresponding changes ... unless you have any other suggestions how to use actionrunner as unprivileged user. Since I see no options, actionrunner HAS TO BE ROOT at the moment.

Of course, then the various install methods (one liner and ansible at least) add stanley with passwordless sudo (skippable with some ansible vars). And then, we get all sorts of side effects of running as root instead of another user.

Now I'm seeing root owned files all over the place as my actions edit or modify files. A file owned by root has many potential actors (any service running as root). But stanley is a stackstorm user, so looking at StackStorm's history is a good way to see why something may have been edited versus reviewing other logs first to see if it was some other service. It may not be more secure to run as stanley, but it is slightly more auditable.

Especially problematic is when an action edits a file that happens to be on our NFS v3 NAS. See, all users have the same access to the NAS as a system mounted drive except root who has special NFS permissions. All other users get mapped to a single NAS user, but root is root. So suddenly, files that are supposed to be world readable/writable are suddenly locked and only root can edit them. It can take a bit to reset those permissions so that they are again accessible correctly over both NFS (and to complicate it even further, through CIFS).

And then, there's all of the sshconfig that I have to add to the root user. Yes, there's passwordless sudo on that special system user, but I would rather not attach all of that config to the root user.

So, maybe actionrunner can continue running as root, but it should drop into a user (configured in st2.conf) to actually run the actions. That user could default to stanley. Or there needs to be a documented/supported way to configure actionrunner to run as stanley. The PR I found is from 2016, there might be other newer reasons (that I'm unware of) that actionrunner needs to run as root. I don't know if there is anything else that will break if I, for example, drop in a systemd unit conf file that overrides the user for the actionrunner service.

In sum, I have two conflicting suggestions to fix this:

1. Make actionrunner run users as a non-root user by default (like stanley), but keep running actionrunner as root.
2. Document/support a method for configuring actionrunner to run as a non-root user instead of the default of root.

[guzzijones]

why does actionrunner needs root permissions? I dived into the links here and all I see is because another user without a password and all the permissions is just as bad. I would argue that a non root user wouldnt have full OS access. Please point me to the specific requirement thanks

[guzzijones]

It looks like the sudo action runner is the holdup. I would argue that we should document a systemd conf fix and note sudo could then be disabled

[docbyte86]

Hello all, is there any update on this? Or do we have a guide of how to setup stackstorm without root permissions?