search

ManageIQ / manageiq-providers-embedded_terraform

ManageIQ plugin for the Embedded Terraform provider.
Apache License 2.0
0 stars 10 forks source link

Add Opentofu Worker #2

Closed agrare closed 4 months ago

agrare commented 6 months ago

Adds an opentofu worker using the embedded_terraform role, note this will be the first non-ruby worker we'll have :tada:

TODO:

Dependents:

agrare commented 6 months ago 
irb(main):001:0> OpentofuWorker.count
=> 1
irb(main):002:0> OpentofuWorker.first
=> 
#<OpentofuWorker:0x00007efe07752b08
 id: 33,
 guid: "30747dc1-40ae-4d3c-a3fd-29169d079f81",
 status: "started",
 started_on: nil,
 stopped_on: nil,
 last_heartbeat: Wed, 06 Mar 2024 21:45:41.655961000 UTC +00:00,
 pid: nil,
 queue_name: nil,
 type: "OpentofuWorker",
 percent_memory: 0.08,
 percent_cpu: 0.02,
 cpu_time: 1470.0,
 os_priority: 20,
 memory_usage: 300175360,
 memory_size: 486223872,
 uri: nil,
 miq_server_id: 1,
 sql_spid: nil,
 proportional_set_size: 285480000,
 unique_set_size: 284116000,
 system_uid: "opentofu-runner.service">
[root@manageiq-devel vmdb]# systemctl status opentofu-runner
● opentofu-runner.service
     Loaded: loaded (/usr/lib/systemd/system/opentofu-runner.service; enabled; preset: disabled)
    Drop-In: /usr/lib/systemd/system/opentofu-runner.service.d
             └─override.conf
     Active: active (running) since Wed 2024-03-06 16:45:43 EST; 35s ago
   Main PID: 9108 (ruby)
      Tasks: 1 (limit: 23130)
     Memory: 18.2M (high: 1.0G available: 1005.7M)
        CPU: 87ms
     CGroup: /system.slice/opentofu-runner.service
             └─9108 /usr/bin/ruby -e sleep

Mar 06 16:45:43 manageiq-devel.localdomain systemd[1]: Started opentofu-runner.service.

Need to handle systemd heartbeat: Switching to Type=simple fixes this. We'll need to adjust when the actual loopback API is running.

agrare commented 5 months ago

Okay I have the opentofu-runner.service being started by EvmServer via the OpentofuWorker class, passing the database password via a secret (we can add whatever other secret info we want here database password just a proof of concept)

[root@manageiq-devel vmdb]# systemctl status opentofu-runner.service
● opentofu-runner.service
     Loaded: loaded (/usr/lib/systemd/system/opentofu-runner.service; enabled; preset: disabled)
     Active: active (running) since Thu 2024-04-04 10:44:30 EDT; 32s ago
    Process: 11438 ExecStartPre=/bin/rm -f /tmp/opentofu-runner.service.cid (code=exited, status=0/SUCCESS)
   Main PID: 11439 (podman)
      Tasks: 28 (limit: 23124)
     Memory: 83.9M
        CPU: 2.709s
     CGroup: /manageiq.slice/opentofu-runner.service
             ├─ 6254 catatonit -P
             ├─11439 /usr/bin/podman run --conmon-pidfile /tmp/opentofu-runner.pid --cidfile /tmp/opentofu-runner.cid --cgroup-manager=cgroupfs --cgroups=no-conmon --log-driver=journald --root=/var/www/miq/vmdb/data/conta>
             ├─11448 /usr/bin/podman run --conmon-pidfile /tmp/opentofu-runner.pid --cidfile /tmp/opentofu-runner.cid --cgroup-manager=cgroupfs --cgroups=no-conmon --log-driver=journald --root=/var/www/miq/vmdb/data/conta>
             ├─11645 /usr/bin/slirp4netns --disable-host-loopback --mtu=65520 --enable-sandbox --enable-seccomp --enable-ipv6 -c -r 3 -e 4 --netns-type=path /tmp/podman-run-987/netns/netns-863ba440-0042-3999-b3b4-ec904858>
             ├─11648 /usr/bin/conmon --api-version 1 -c c276456a86cc1b9172db81003a15605d7458e9a9f9310010c8f8c142264e2a6b -u c276456a86cc1b9172db81003a15605d7458e9a9f9310010c8f8c142264e2a6b -r /usr/bin/crun -b /var/www/miq>
             └─11650 ruby /usr/src/app/./opentofu-runner.rb

Apr 04 10:44:32 manageiq-devel.localdomain podman[11448]: Copying blob sha256:5f74a64ac7702f1a3cf514af2d28600c186751c703bb08396d01631924d2b5d0
Apr 04 10:44:41 manageiq-devel.localdomain podman[11448]: Copying config sha256:4c26793dbb863d854d7a16a2bb1a31fdc0f9ec49705b508a211ed12eb90f7bc3
Apr 04 10:44:41 manageiq-devel.localdomain podman[11448]: Writing manifest to image destination
Apr 04 10:44:41 manageiq-devel.localdomain podman[11448]: 2024-04-04 10:44:41.350721535 -0400 EDT m=+10.487458035 image pull 4c26793dbb863d854d7a16a2bb1a31fdc0f9ec49705b508a211ed12eb90f7bc3 docker.io/agrare/opentofu-runne>
Apr 04 10:44:41 manageiq-devel.localdomain podman[11448]: 2024-04-04 10:44:41.536073634 -0400 EDT m=+10.672810125 container create c276456a86cc1b9172db81003a15605d7458e9a9f9310010c8f8c142264e2a6b (image=docker.io/agrare/o>
Apr 04 10:44:41 manageiq-devel.localdomain podman[11448]: 2024-04-04 10:44:41.69561559 -0400 EDT m=+10.832352100 container init c276456a86cc1b9172db81003a15605d7458e9a9f9310010c8f8c142264e2a6b (image=docker.io/agrare/open>
Apr 04 10:44:41 manageiq-devel.localdomain podman[11448]: 2024-04-04 10:44:41.700127055 -0400 EDT m=+10.836863545 container start c276456a86cc1b9172db81003a15605d7458e9a9f9310010c8f8c142264e2a6b (image=docker.io/agrare/op>
Apr 04 10:44:41 manageiq-devel.localdomain podman[11448]: 2024-04-04 10:44:41.732550647 -0400 EDT m=+10.869287157 container attach c276456a86cc1b9172db81003a15605d7458e9a9f9310010c8f8c142264e2a6b (image=docker.io/agrare/o>
Apr 04 10:44:41 manageiq-devel.localdomain opentofu-runner[11648]: {"DATABASE_PASSWORD":"smartvm"}
Apr 04 10:44:41 manageiq-devel.localdomain podman[11448]: {"DATABASE_PASSWORD":"smartvm"}
agrare commented 5 months ago

For the systemd service I broadly followed https://www.redhat.com/sysadmin/podman-run-pods-systemd-services without some of the complexity of creating a Pod plus a Container

miq-bot commented 4 months ago

Checked commits https://github.com/agrare/manageiq-providers-embedded_terraform/compare/fc971559e050be6e633203aafa440de77f2d26e1~...4e6881afdc9fa560a2a68abae5daf062b29c8259 with ruby 2.7.8, rubocop 1.56.3, haml-lint 0.51.0, and yamllint 2 files checked, 0 offenses detected Everything looks fine. :trophy:

agrare commented 4 months ago

TODO in a follow-up, check Terraform::Runner.available? and prevent the worker from starting up if it isn't available

agrare commented 4 months ago

TODO check if you can use an Environment Variable in an ExecStart to change the runner image label