search

RedHatGov / redhatgov.workshops

This is a collection of Ansible-deployed workshop environments. Use it in combination with the student workbook content, from the repo at https://github.com/RedHatGov/redhatgov.github.io
http://redhatgov.io
64 stars 69 forks source link

RHEL8 workshop Integrate container into systemd fails #140

Closed dpullman-emergent closed 3 years ago

dpullman-emergent commented 3 years ago

In Ex 1.8, Section 5: Use skopeo and podman to integrate the container into systemd, when you get to Step 3: Integrate container into systemd, the command to enable and start the container-web.service fails to start.

[ec2-user@ip-10-0-2-184 ~]$ sudo systemctl enable --now container-web.service
Created symlink /etc/systemd/system/multi-user.target.wants/container-web.service → /etc/systemd/system/container-web.service.
Created symlink /etc/systemd/system/default.target.wants/container-web.service → /etc/systemd/system/container-web.service.
Job for container-web.service failed because the control process exited with error code.
See "systemctl status container-web.service" and "journalctl -xe" for details.

It appears that because fapolicyd is enabled in the OSPP profile for OpenSCAP, the system usage of containers is blocked.

If we turn off fapolicyd and try again, the container starts.

[ec2-user@ip-10-0-2-184 ~]$ sudo systemctl status fapolicyd
● fapolicyd.service - File Access Policy Daemon
   Loaded: loaded (/usr/lib/systemd/system/fapolicyd.service; enabled; vend>
   Active: active (running) since Thu 2021-03-04 16:07:02 UTC; 38min ago
 Main PID: 41340 (fapolicyd)
    Tasks: 4 (limit: 10899)
   Memory: 76.1M
   CGroup: /system.slice/fapolicyd.service
           └─41340 /usr/sbin/fapolicyd
[ec2-user@ip-10-0-2-184 ~]$ sudo systemctl stop fapolicyd
[ec2-user@ip-10-0-2-184 ~]$ sudo systemctl enable --now container-web.service
[ec2-user@ip-10-0-2-184 ~]$ sudo systemctl status container-web.service 
● container-web.service - Podman container-web.service
   Loaded: loaded (/etc/systemd/system/container-web.service; enabled; vend>
   Active: active (running) since Thu 2021-03-04 16:46:09 UTC; 15s ago
     Docs: man:podman-generate-systemd(1)
  Process: 46448 ExecStart=/usr/bin/podman start web (code=exited, status=0>
 Main PID: 46544 (conmon)
    Tasks: 2 (limit: 10899)
   Memory: 3.6M
   CGroup: /system.slice/container-web.service
           └─46544 /usr/bin/conmon --api-version 1 -c 168b5c70e24f064894b7

Separately, the container-web.service shows the following:

Error: unable to start container "168b5c70e24f064894b7f2c7b869bcd81c6ba28e31a9182dfa87ae0220e5dabc": /usr/bin/runc: error while loading shared libraries: libpthread.so.0: cannot open shared object file: Operation not permitted: OCI permission denied

We found this open bugzilla issue Bug 1907870 - cannot run podman in 8.3 (https://bugzilla.redhat.com/show_bug.cgi?id=1907870) that seems to be current status on this.

We have a workshop with a customer next week. We can explain the issue during the workshop, but if there is any suggested workaround other than that, please let me know.

ajacocks commented 3 years ago

David,

Sorry for the late reply. We haven't been running this workshop on OSPP-secured hosts, so we hadn't run into this issue. I'll keep an eye on this bug.

Thanks!

On Thu, Mar 4, 2021 at 12:46 PM David G. Pullman @.***> wrote:

In Ex 1.8, Section 5: Use skopeo and podman to integrate the container into systemd, when you get to Step 3: Integrate container into systemd, the command to enable and start the container-web.service fails to start.

@.*** ~]$ sudo systemctl enable --now container-web.service

Created symlink /etc/systemd/system/multi-user.target.wants/container-web.service → /etc/systemd/system/container-web.service.

Created symlink /etc/systemd/system/default.target.wants/container-web.service → /etc/systemd/system/container-web.service.

Job for container-web.service failed because the control process exited with error code.

See "systemctl status container-web.service" and "journalctl -xe" for details.

It appears that because fanotifyd is enabled in the OSPP profile for OpenSCAP, the system usage of containers is blocked.

If we turn off fanotifyd and try again, the container starts.

@.*** ~]$ sudo systemctl status fapolicyd

● fapolicyd.service - File Access Policy Daemon

Loaded: loaded (/usr/lib/systemd/system/fapolicyd.service; enabled; vend>

Active: active (running) since Thu 2021-03-04 16:07:02 UTC; 38min ago

Main PID: 41340 (fapolicyd)

Tasks: 4 (limit: 10899)

Memory: 76.1M

CGroup: /system.slice/fapolicyd.service

       └─41340 /usr/sbin/fapolicyd

@.*** ~]$ sudo systemctl stop fapolicyd

@.*** ~]$ sudo systemctl enable --now container-web.service

@.*** ~]$ sudo systemctl status container-web.service

● container-web.service - Podman container-web.service

Loaded: loaded (/etc/systemd/system/container-web.service; enabled; vend>

Active: active (running) since Thu 2021-03-04 16:46:09 UTC; 15s ago

 Docs: man:podman-generate-systemd(1)

Process: 46448 ExecStart=/usr/bin/podman start web (code=exited, status=0>

Main PID: 46544 (conmon)

Tasks: 2 (limit: 10899)

Memory: 3.6M

CGroup: /system.slice/container-web.service

       └─46544 /usr/bin/conmon --api-version 1 -c 168b5c70e24f064894b7

Separately, the container-web.service shows the following:

Error: unable to start container "168b5c70e24f064894b7f2c7b869bcd81c6ba28e31a9182dfa87ae0220e5dabc": /usr/bin/runc: error while loading shared libraries: libpthread.so.0: cannot open shared object file: Operation not permitted: OCI permission denied

We found this open bugzilla issue Bug 1907870 - cannot run podman in 8.3 ( https://bugzilla.redhat.com/show_bug.cgi?id=1907870) that seems to be current status on this.

We have a workshop with a customer next week. We can explain the issue during the workshop, but if there is any suggested workaround other than that, please let me know.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/RedHatGov/redhatgov.workshops/issues/140, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFLVO4TNEWP56ZHANPNMP7DTB7BOFANCNFSM4YTWGY3Q .

--

Alexander Jacocks, RHCE | Staff Solution Architect (e) @.*** | Public Sector Business Development (c) (240) 447-5974 | Tyson's Corner, VA (g) CC28 8136 C253 907A B9BB 9EEA D3C8 1A3A 790A 0AB9

dpullman-emergent commented 3 years ago

Alex, actually after completing section 4 of exercise 1.7, you have applied OSPP to the instance. That includes fapolicyd.

We could add a step at the beginning of section 5 of 1.8 to stop fapolicyd, with a statement that it's just for this part of the exercise? If that would be alright I can submit a change for that.

Thanks! --David

ajacocks commented 3 years ago

David,

Yeah, you are indeed correct! I just went through and verified all of this, and I'm surprised that we hadn't run into this, before.

Please, indeed, submit that change as a PR, and I'll merge it.

Thanks for finding that.

On Wed, Mar 17, 2021 at 1:51 PM David G. Pullman @.***> wrote:

Alex, actually after completing section 4 of exercise 1.7, you have applied OSPP to the instance. That includes fapolicyd.

We could add a step at the beginning of section 5 of 1.8 to stop fapolicyd, with a statement that it's just for this part of the exercise? If that would be alright I can submit a change for that.

Thanks! --David

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RedHatGov/redhatgov.workshops/issues/140#issuecomment-801287423, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFLVO4QZKQL2TPJGSQXXRCDTEDT3NANCNFSM4YTWGY3Q .

--

Alexander Jacocks, RHCE | Staff Solution Architect (e) @.*** | Public Sector Business Development (c) (240) 447-5974 | Tyson's Corner, VA (g) CC28 8136 C253 907A B9BB 9EEA D3C8 1A3A 790A 0AB9

dpullman-emergent commented 3 years ago

This issue was resolved when #375 was merged. Closing.