PCluster expects to execute scripts out of /tmp

[dan-schlecht]

The ParallelCluster team uses this template to report known issues on github. If you are reporting an issue, please use the 'Bug report' template instead.

Bug description

Provide the following information:

• A clear and concise description of the bug and its impact on ParallelCluster customers
• Steps to reproduce the bug, including an example configuration file
• Versions of the various components (CLI package, node daemons, cookbook, etc.) affected by the bug
• Workaround customers can use until a fix is in place, or the fix itself if it's already done

Due to security concerns, all of our corporate AMIs do not allow script execution from /tmp. Thus, create and createami commands fail. As a workaround for create, I download and update yaml templates files to update user data sections for master and fleet servers. I also have to download and update the main json stack file to point to my new Master server sub-stack file. I also have to use the undocumented config parameter hit-template to point to my local fleet substack file

Would it be possible to have a config options to point to bash scripts to add to the beginning of user data sections of master server and fleet substacks? Then, I could easily add commands to remount /tmp to allow execute and add environment variables to fix SSL trust issues.

[dan-schlecht]

Another option could be for pcluster code to remount /tmp to allow for execution when cluster creation needs execution rights.

Does anywhere else in the pcluster design/implementation require /tmp execution permissions? Like during runtime?

[tilne]

do not allow script execution from /tmp

By this do you mean it doesn't allow for executing anything under /tmp, or do you mean that they don't allow anything to be executed when a process's current working directory is under /tmp?

Another option could be for pcluster code to remount /tmp to allow for execution when cluster creation needs execution rights.

What is the benefit of un-mounting and then re-mounting /tmp? Wouldn't the restriction against executing things out of /tmp still be in place?

Does anywhere else in the pcluster design/implementation require /tmp execution permissions? Like during runtime?

I don't see anything in our source that would rely on it after a node has been successfully booted, but I can't guarantee that none of our dependencies do.

Would it be possible to have a config options to point to bash scripts to add to the beginning of user data sections of master server and fleet substacks?

I think this type of use case is what the pre- and post-install scripts are intended to address. That's not to say that it addresses your use case. I'll discuss this request with the team.

[dan-schlecht]

the VM's configuration doesn't allow any executables from the /tmp partition. For example, I can execute a bash script from my home directory. However, if I copy the same script to the /tmp directory, the bash script will not execute even if the the script permissions are 777. For security reasons, it is becoming more common to not allow executables on the /tmp partition.

My workaround is to remount /tmp to allow executables with this command: mount -o remount,exec /tmp

However, if the VM is rebooted /tmp will come up with noexec again. So, I had to add an init.d script to remount /tmp with exec. Since the /tmp partition is created with noexec, this is the only workaround I have.

I have tried a pre-install script, but my script doesn't get executed before the failure happens. I guess the cloud-init and cfn-init stuff is executed before my pre-install script. This is why I had to update the user data of the master substack and fleet substack template files. I was hoping for at least one of the following to make an easier workaround:

1. AWS ParallelCluster doesn't require executables from /tmp or a config parameter to point to another temporary folder
2. A way to easily add to Master and Fleet userdata substacks. This would be similar to pre-install and post-install scripts, but looks like userdata section of template is executed before the currently supported pre-install script
3. Pcluster code remounts /tmp with execution permission as needed when need execute permissions: mount -o remount,exec /tmp when no longer need execute: mount -o remount,noexec /tmp

[yuleiwan]

Followup in Createami fails when trying to createSecurityGroup #2917