exec user process caused "operation not permitted" due to OPTIONS='--selinux-enabled'

[louwersj]

Did a clean install of Fn Project based upon the documentation provided by the Fn project. Using the Oracle provided ol73 vagrant image.

Invoking a function provides the following error: container failed to initialize, please ensure you are using the latest fdk / format and check the logs.

The fn logs show the following:

[time="2019-05-04T12:20:29Z" level=info msg="hot function terminated" app_id=01DA1A2VPENG8G00GZJ0000001 container_id=01DA1CKPXRNG8G00GZJ000000Q cpus= error="container exit code 1" fn_id=01DA1A5YC6NG8G00GZJ0000002 idle_timeout=30 image="fndemouser/app1:0.0.3" memory=256
time="2019-05-04T12:20:29Z" level=error msg="api error" action="server.handleHTTPTriggerCall)-fm" app_name=app1 code=502 error="container failed to initialize, please ensure you are using the latest fdk / format and check the logs" trigger_source=/app1]

Using a remote syslog server "papertrailapp.com" I was able to capture the following error: May 04 14:44:02 fnproject app_id=01DA1A2VPENG8G00GZJ0000001,fn_id=01DA1A5YC6NG8G00GZJ0000002: standard_init_linux.go:190: exec user process caused "operation not permitted"

After searching I found the following workaround, after applying it the functions are working as expected.

1) review /etc/sysconfig/docker 2) remove --selinux-enabled from OPTIONS 3) restart Docker and FN 4) test function (should work now)

Even though it resolves the issue at hand , it feels like a workaround at the best. Either, the documentation should mention this as a required step (to check) or we should find a way to prevent this situation from happening.

background version info:

[root@fnproject app1]# fn version
Client version is latest version: 0.5.76
Server version:  0.3.702

[root@fnproject app1]# docker --version
Docker version 18.09.1-ol, build c3ab8a8

[root@fnproject app1]# uname -a
Linux fnproject 4.1.12-61.1.28.el7uek.x86_64 #2 SMP Thu Feb 23 19:55:12 PST 2017 x86_64 x86_64 x86_64 GNU/Linux

[root@fnproject app1]# sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      29

[samueletonon]

Got same problem while following python tutorial.

Official output from fn invoke is:

fn invoke pythonapp pythonfn Error invoking function. status: 502 message: container failed to initialize, please ensure you are using the latest fdk / format and check the logs

however with papertrail syslog i get:

Jun 22 22:10:40 fuffa app_id=01DE0BH5PNNG8G00GZJ0000001,fn_id=01DE0BJZTNNG8G00GZJ0000002: standard_init_linux.go:190: exec user process caused "operation not permitted"

this using docker from ubunt 18.04 with snap. if i use docker-ce with normal installation no problems at all

[louwersj]

@samueletonon , did you try the workaround I proposed in the original post? Even though it is a workaround it can solve the issue and can help you move forward.

1. review /etc/sysconfig/docker
2. remove --selinux-enabled from OPTIONS
3. restart Docker and FN

[samueletonon]

hello, no i can't use your workaround as snap install everything in a different way. for example in the machine i am using selinux has been disabled, thus if i install normal docker through apt instead of snap the issue is not present -- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[louwersj]

@samueletonon interesting..... that would / could mean that the way the installer is setting some config differs per package. No real surprise, however, it would be good to figure out what exactly. I did all the installs with yum (no apt/snap).

[rohitcelestial]

I just removed the snap docker and reinstalled with apt get and that resolved the issue.