PermissionError: [Errno 13] Permission denied: '/home/ansible/wfexs-backend-test/wf-cache'

[dcl10]

Description

When staging a workflow for the first time as non-sudo/root user I'm getting the error in the title. Oddly, deleting the cache dir and re-running the stage command seems to fix the issue. As does adding write permissions with chown. I'm not sure if it has anything to do with the calls to os.makesirs or a umask issue.

Here's something I was reading about the problem. Not sure if it will be helpful. https://stackoverflow.com/questions/5231901/permission-problems-when-creating-a-dir-with-os-makedirs-in-python/67723702#67723702

[jmfernandez]

In order to double check I have been having a look at the different points where directories are created, and as I was remembering no one of the os.makedirs calls explicitly set the directory permissions. So, the permissions used before applying the umask for the directory creation are the default ones, described at https://docs.python.org/3/library/os.html#os.makedirs

So, my guess is that the initial umask inherited by the process calling WfExS is what it can be troublesome in the first call.

[dcl10]

@jmfernandez, I've done some reading and a little playing around. I think the best thing to do is: in the places where you are calling os.makedirs, you should explicitly set the mode parameter using an octal number. e.g. 0o777 for all permissions for everyone, or 0o755 for everything for you and r+x for everyone else. Or more conservatively 0o644 for r+w for you and r only for everyone else. The choice depends on what the directories are used for.

[jmfernandez]

Hi @dcl10 , the real point there is that umask is a property from any process, inheriting its initial value from the parent process which forked the program. You can see the effect of the different umask values using the very same mode of 0o777:

jmfernandez@pavonis[1]:/tmp> mkdir playground
jmfernandez@pavonis[2]:/tmp> cd playground
jmfernandez@pavonis[3]:/tmp/playground> umask
0022
jmfernandez@pavonis[4]:/tmp/playground> python -c 'import os; os.makedirs("default_mask", mode=0o777)'
jmfernandez@pavonis[5]:/tmp/playground> ls -ld
drwxr-xr-x 1 jmfernandez users 24 ago 25 19:08 .
jmfernandez@pavonis[6]:/tmp/playground> ls -ld *
drwxr-xr-x 1 jmfernandez users 0 ago 25 19:08 default_mask
jmfernandez@pavonis[7]:/tmp/playground> umask 0222
jmfernandez@pavonis[8]:/tmp/playground> umask
0222
jmfernandez@pavonis[9]:/tmp/playground> python -c 'import os; os.makedirs("ro_mask", mode=0o777)'
jmfernandez@pavonis[10]:/tmp/playground> ls -ld *
drwxr-xr-x 1 jmfernandez users 0 ago 25 19:08 default_mask
dr-xr-xr-x 1 jmfernandez users 0 ago 25 19:08 ro_mask
jmfernandez@pavonis[11]:/tmp/playground> umask 0077
jmfernandez@pavonis[12]:/tmp/playground> python -c 'import os; os.makedirs("useronly_mask", mode=0o777)'
jmfernandez@pavonis[13]:/tmp/playground> ls -ld *
drwxr-xr-x 1 jmfernandez users 0 ago 25 19:08 default_mask
dr-xr-xr-x 1 jmfernandez users 0 ago 25 19:08 ro_mask
drwx------ 1 jmfernandez users 0 ago 25 19:09 useronly_mask
jmfernandez@pavonis[14]:/tmp/playground> umask 0777
jmfernandez@pavonis[15]:/tmp/playground> python -c 'import os; os.makedirs("noone_mask", mode=0o777)'
jmfernandez@pavonis[16]:/tmp/playground> ls -ld *
drwxr-xr-x 1 jmfernandez users 0 ago 25 19:08 default_mask
d--------- 1 jmfernandez users 0 ago 25 19:10 noone_mask
dr-xr-xr-x 1 jmfernandez users 0 ago 25 19:08 ro_mask
drwx------ 1 jmfernandez users 0 ago 25 19:09 useronly_mask

The mask from the umask property is applied by the operating system when a file or directory creation operation is requested from the process. It is done so in order to give control to the user over the default permissions without having to change every program.

Of course, a process is allowed to change its own umask property value, as you can see:

jmfernandez@pavonis[1]:/tmp> mkdir playground2
jmfernandez@pavonis[2]:/tmp> cd playground2
jmfernandez@pavonis[3]:/tmp/playground2> umask
0022
jmfernandez@pavonis[4]:/tmp/playground2> umask 0777
jmfernandez@pavonis[5]:/tmp/playground2> python -c 'import os; os.makedirs("noone_mask", mode=0o777)'
jmfernandez@pavonis[6]:/tmp/playground2> python -c 'import os; os.umask(0o000); os.makedirs("everyone_mask", mode=0o777)'
jmfernandez@pavonis[7]:/tmp/playground2> ls -ld *
drwxrwxrwx 1 jmfernandez users 0 ago 25 19:23 everyone_mask
d--------- 1 jmfernandez users 0 ago 25 19:23 noone_mask

Typical programs changing its umask are shells (like bash), daemons and services. Outside these scenarios, it is considered an antipattern that a program changes its own umask, as then the user cannot tweak the permissions through the initially inherited umask.

So, my recommendation is that the umask should be adjusted just before WfExS-backend is run. For instance to something like umask 0077 or umask 0027, depending on whether you trust the default group of the user.