kubearmor / packer-plugin-kubearmor

KubeArmor Packer Provider
Mozilla Public License 2.0
0 stars 3 forks source link

kubearmor packer provisioner #3

Closed Prateeknandle closed 1 year ago

Prateeknandle commented 1 year ago

Includes:

  1. Kubearmor recommend
  2. Ansible playbook for installing kubearmor, karmor and dependencies
  3. Integration with packer
ShubhamTatvamasi commented 1 year ago

I tested kubearmor packer plugin with Proxmox with code below. But I am getting error at the time of Gathering Facts when running Ansible playbook. This can be a macOS issue, I will test the same with Linux system and update.

https://github.com/Prateeknandle/packer-plugin-kubearmor/pull/1/files

➜  ubuntu-server git:(packer-provisioner) ✗ packer build .
proxmox-iso.ubuntu-server: output will be in this color.

==> proxmox-iso.ubuntu-server: Creating VM
==> proxmox-iso.ubuntu-server: No VM ID given, getting next free from Proxmox
==> proxmox-iso.ubuntu-server: Starting VM
==> proxmox-iso.ubuntu-server: Starting HTTP server on port 8021
==> proxmox-iso.ubuntu-server: Waiting 10s for boot
==> proxmox-iso.ubuntu-server: Typing the boot command
==> proxmox-iso.ubuntu-server: Waiting for SSH to become available...
==> proxmox-iso.ubuntu-server: Connected to SSH!
==> proxmox-iso.ubuntu-server: Provisioning with shell script: /var/folders/rk/z7t2cqnn6xx5qrzcm2pfqkxh0000gn/T/packer-shell1370647593
==> proxmox-iso.ubuntu-server: Provisioning with Ansible...
    proxmox-iso.ubuntu-server: Setting up proxy adapter for Ansible....
==> proxmox-iso.ubuntu-server: Executing Ansible: ansible-playbook -e packer_build_name="*****-server" -e packer_builder_type=proxmox-iso -e packer_http_addr=192.168.1.22:8021 --ssh-extra-args '-o IdentitiesOnly=yes' -e ansible_ssh_private_key_file=/var/folders/rk/z7t2cqnn6xx5qrzcm2pfqkxh0000gn/T/ansible-key1632853151 -i /var/folders/rk/z7t2cqnn6xx5qrzcm2pfqkxh0000gn/T/packer-provisioner-ansible633370001 /Users/shubhamtatvamasi/myfiles/git/packer-plugin-kubearmor/ansible/conf.yml
    proxmox-iso.ubuntu-server:
    proxmox-iso.ubuntu-server: PLAY [Install kubearmor] *******************************************************
    proxmox-iso.ubuntu-server:
    proxmox-iso.ubuntu-server: TASK [Gathering Facts] *********************************************************
    proxmox-iso.ubuntu-server: fatal: [default]: FAILED! => {"msg": "failed to transfer file to /Users/shubhamtatvamasi/.ansible/tmp/ansible-local-943704cpejshi/tmp_ukot1gt ~shubhamtatvamasi/.ansible/tmp/ansible-tmp-1694772637.448343-94373-233009813607689/AnsiballZ_setup.py:\n\nscp: dest open \"'~shubhamtatvamasi/.ansible/tmp/ansible-tmp-1694772637.448343-94373-233009813607689/AnsiballZ_setup.py'\": No such file or directory\r\nscp: failed to upload file /Users/shubhamtatvamasi/.ansible/tmp/ansible-local-943704cpejshi/tmp_ukot1gt to '~shubhamtatvamasi/.ansible/tmp/ansible-tmp-1694772637.448343-94373-233009813607689/AnsiballZ_setup.py'\r\n"}
    proxmox-iso.ubuntu-server:
    proxmox-iso.ubuntu-server: PLAY RECAP *********************************************************************
    proxmox-iso.ubuntu-server: default                    : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
    proxmox-iso.ubuntu-server:
==> proxmox-iso.ubuntu-server: Provisioning step had errors: Running the cleanup provisioner, if present...
==> proxmox-iso.ubuntu-server: Stopping VM
==> proxmox-iso.ubuntu-server: Deleting VM
Build 'proxmox-iso.ubuntu-server' errored after 4 minutes 934 milliseconds: Error executing Ansible: Non-zero exit status: exit status 2

==> Wait completed after 4 minutes 934 milliseconds

==> Some builds didn't complete successfully and had errors:
--> proxmox-iso.ubuntu-server: Error executing Ansible: Non-zero exit status: exit status 2

==> Builds finished but no artifacts were created.
ShubhamTatvamasi commented 1 year ago

I tried running packer build from an Ubuntu VM, but the image is getting stuck on boot. I will have to try something else.

Screenshot 2023-09-15 at 10 33 38 PM
rksharma95 commented 1 year ago

i'm able to test this PR using virtualbox vm running ubuntu 20.04

$ packer build config.pkr.hcl
``` Warning: Bundled plugins used This template relies on the use of plugins bundled into the Packer binary. The practice of bundling external plugins into Packer will be removed in an upcoming version. To remove this warning, add the following section to your template: packer { required_plugins { ansible = { source = "github.com/hashicorp/ansible" version = "~> 1" } } } Then run 'packer init' to manage installation of the plugins learn-packer.virtualbox-ovf.basic-example: output will be in this color. ==> learn-packer.virtualbox-ovf.basic-example: Retrieving Guest additions ==> learn-packer.virtualbox-ovf.basic-example: Trying /usr/share/virtualbox/VBoxGuestAdditions.iso ==> learn-packer.virtualbox-ovf.basic-example: Trying /usr/share/virtualbox/VBoxGuestAdditions.iso ==> learn-packer.virtualbox-ovf.basic-example: /usr/share/virtualbox/VBoxGuestAdditions.iso => /usr/share/virtualbox/VBoxGuestAdditions.iso ==> learn-packer.virtualbox-ovf.basic-example: Retrieving OVF/OVA ==> learn-packer.virtualbox-ovf.basic-example: Trying ubuntu.ova ==> learn-packer.virtualbox-ovf.basic-example: Trying ubuntu.ova ==> learn-packer.virtualbox-ovf.basic-example: ubuntu.ova => /home/hp/Documents/packer-plugin-kubearmor/ubuntu.ova ==> learn-packer.virtualbox-ovf.basic-example: Importing VM: /home/hp/Documents/packer-plugin-kubearmor/ubuntu.ova ==> learn-packer.virtualbox-ovf.basic-example: Mounting ISOs... learn-packer.virtualbox-ovf.basic-example: No ISOs to mount; continuing... ==> learn-packer.virtualbox-ovf.basic-example: Creating forwarded port mapping for communicator (SSH, WinRM, etc) (host port 2361) ==> learn-packer.virtualbox-ovf.basic-example: Starting the virtual machine... ==> learn-packer.virtualbox-ovf.basic-example: Waiting 10s for boot... ==> learn-packer.virtualbox-ovf.basic-example: Typing the boot command... ==> learn-packer.virtualbox-ovf.basic-example: Using SSH communicator to connect: 127.0.0.1 ==> learn-packer.virtualbox-ovf.basic-example: Waiting for SSH to become available... ==> learn-packer.virtualbox-ovf.basic-example: Connected to SSH! ==> learn-packer.virtualbox-ovf.basic-example: Uploading VirtualBox version info (7.0.4) ==> learn-packer.virtualbox-ovf.basic-example: Uploading VirtualBox guest additions ISO... ==> learn-packer.virtualbox-ovf.basic-example: Provisioning with Ansible... learn-packer.virtualbox-ovf.basic-example: Setting up proxy adapter for Ansible.... ==> learn-packer.virtualbox-ovf.basic-example: Executing Ansible: ansible-playbook -e packer_build_name="basic-example" -e packer_builder_type=virtualbox-ovf -e packer_http_addr=10.0.2.2:0 --ssh-extra-args '-o IdentitiesOnly=yes' -e ansible_ssh_private_key_file=/tmp/ansible-key4178117464 -i /tmp/packer-provisioner-ansible3580934950 /home/hp/Documents/packer-plugin-kubearmor/ansible/conf.yml learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: PLAY [Install kubearmor] ******************************************************* learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Gathering Facts] ********************************************************* learn-packer.virtualbox-ovf.basic-example: ok: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [include_vars] ************************************************************ learn-packer.virtualbox-ovf.basic-example: ok: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Ensure curl] ************************************************************* learn-packer.virtualbox-ovf.basic-example: ok: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Ensure build-essential] ************************************************** learn-packer.virtualbox-ovf.basic-example: ok: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Ensure libelf-dev] ******************************************************* learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Ensure pkg-config] ******************************************************* learn-packer.virtualbox-ovf.basic-example: ok: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Ensure net-tools] ******************************************************** learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [run uname -r] ************************************************************ learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Ensure linux-headers-generic] ******************************************** learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Ensure linux-headers] **************************************************** learn-packer.virtualbox-ovf.basic-example: ok: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Ensure linux-tools] ****************************************************** learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Ensure clang] ************************************************************ learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Ensure llvm] ************************************************************* learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Ensure bpfcc-tools] ****************************************************** learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Create tmp folder] ******************************************************* learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Check RPM] *************************************************************** learn-packer.virtualbox-ovf.basic-example: fatal: [default]: FAILED! => {"changed": false, "cmd": "rpm --help", "msg": "[Errno 2] No such file or directory: b'rpm'", "rc": 2, "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} learn-packer.virtualbox-ovf.basic-example: ...ignoring learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Check dpkg] ************************************************************** learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Download dpkg] *********************************************************** learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Download RPM] ************************************************************ learn-packer.virtualbox-ovf.basic-example: skipping: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Install using RPM] ******************************************************* learn-packer.virtualbox-ovf.basic-example: skipping: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Install using dpkg] ****************************************************** learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Enable kubearmor service] ************************************************ learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Start kubearmor service] ************************************************* learn-packer.virtualbox-ovf.basic-example: ok: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: TASK [Install karmor] ********************************************************** learn-packer.virtualbox-ovf.basic-example: changed: [default] learn-packer.virtualbox-ovf.basic-example: learn-packer.virtualbox-ovf.basic-example: PLAY RECAP ********************************************************************* learn-packer.virtualbox-ovf.basic-example: default : ok=22 changed=14 unreachable=0 failed=0 skipped=2 rescued=0 ignored=1 learn-packer.virtualbox-ovf.basic-example: ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/write-under-dev-dir ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/pkg-mngr-exec ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/k8s-client-tool-exec ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/maint-tools-access ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/sys-admin-scope-mod ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/network-service-scanning ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/access-ctrl-permission-mod ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/system-owner-discovery ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/user-grp-mod ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/system-network-env-mod ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/cronjob-cfg ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/shell-history-mod ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/write-under-bin-dir ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/write-etc-dir ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/file-integrity-monitoring ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/write-in-shm-dir ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/system-files-mod ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/trusted-cert-mod ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/file-system-mounts ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/remote-services ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/impair-defense ==> learn-packer.virtualbox-ovf.basic-example: file uploaded: /home/vboxuser/policies/remote-file-copy ==> learn-packer.virtualbox-ovf.basic-example: Gracefully halting virtual machine... ==> learn-packer.virtualbox-ovf.basic-example: Preparing to export machine... learn-packer.virtualbox-ovf.basic-example: Deleting forwarded port mapping for the communicator (SSH, WinRM, etc) (host port 2361) ==> learn-packer.virtualbox-ovf.basic-example: Exporting virtual machine... learn-packer.virtualbox-ovf.basic-example: Executing: export packer-basic-example-1695184080 --output output-basic-example/packer-basic-example-1695184080.ovf ==> learn-packer.virtualbox-ovf.basic-example: Cleaning up floppy disk... ==> learn-packer.virtualbox-ovf.basic-example: Deregistering and deleting imported VM... Build 'learn-packer.virtualbox-ovf.basic-example' finished after 21 minutes 35 seconds. ==> Wait completed after 21 minutes 35 seconds ==> Builds finished. The artifacts of successful builds are: --> learn-packer.virtualbox-ovf.basic-example: VM files in directory: output-basic-example ```
vboxuser@ubuntu:~$ sudo systemctl status kubearmor
● kubearmor.service - KubeArmor
     Loaded: loaded (/lib/systemd/system/kubearmor.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-09-20 10:24:57 IST; 4min 53s ago
   Main PID: 594 (kubearmor)
      Tasks: 9 (limit: 2263)
     Memory: 116.1M
     CGroup: /system.slice/kubearmor.service
             └─594 /opt/kubearmor/kubearmor

Sep 20 10:25:05 ubuntu kubearmor[594]: 2023-09-20 10:25:05.403822        WARN        Error while looking for CRI socket file
Sep 20 10:25:05 ubuntu kubearmor[594]: 2023-09-20 10:25:05.404164        WARN        Failed to monitor containers:  is not a supported CRI socket.
Sep 20 10:25:05 ubuntu kubearmor[594]: 2023-09-20 10:25:05.404368        INFO        Using  for monitoring containers
Sep 20 10:25:06 ubuntu kubearmor[594]: 2023-09-20 10:25:06.404647        INFO        Started to monitor host security policies on gRPC
Sep 20 10:25:06 ubuntu kubearmor[594]: 2023-09-20 10:25:06.404695        INFO        Started to serve gRPC-based log feeds
Sep 20 10:25:06 ubuntu kubearmor[594]: 2023-09-20 10:25:06.404707        INFO        Initialized KubeArmor
Sep 20 10:25:06 ubuntu kubearmor[594]: 2023-09-20 10:25:06.404763        WARN        Policies dir not found for restoration
Sep 20 10:27:04 ubuntu kubearmor[594]: 2023-09-20 10:27:04.392948        INFO        Added a new client (6b6eea3a-c09f-4af1-b885-f1ad5fc37552, policy) for WatchAlerts
Sep 20 10:27:47 ubuntu kubearmor[594]: 2023-09-20 10:27:47.784275        INFO        Detected a Host Security Policy (added/pkg-mngr-exec)
Sep 20 10:27:47 ubuntu kubearmor[594]: 2023-09-20 10:27:47.894093        INFO        Updated 31 host security rules to the KubeArmor host profile in ubuntu
vboxuser@ubuntu:~$ karmor logs --gRPC=:32767&
[1] 1979
vboxuser@ubuntu:~$ Created a gRPC client (:32767)
Checked the liveness of the gRPC server
Started to watch alerts

vboxuser@ubuntu:~$ apt
== Alert / 2023-09-20 05:00:19.098045 ==
HostName: ubuntu
Type: MatchedHostPolicy
PolicyName: pkg-mngr-exec
Severity: 5
Message: Alert! Execution of package management process inside container is denied
Source: /usr/bin/bash
Resource: /usr/bin/apt
Operation: Process
Action: Block
Data: syscall=SYS_EXECVE
Enforcer: AppArmor
Result: Permission denied
ATags: [NIST NIST_800-53_CM-7(4) SI-4 process NIST_800-53_SI-4]
HostPID: 1986
HostPPID: 1943
PID: 1986
PPID: 1943
ParentProcessName: /usr/bin/bash
ProcessName: /usr/bin/apt
Tags: NIST,NIST_800-53_CM-7(4),SI-4,process,NIST_800-53_SI-4
UID: 1000
-bash: /usr/bin/apt: Permission denied
Prateeknandle commented 1 year ago

thanks @rksharma95 for testing

ShubhamTatvamasi commented 1 year ago

i'm able to test this PR using virtualbox vm running ubuntu 20.04

$ packer build config.pkr.hcl

vboxuser@ubuntu:~$ sudo systemctl status kubearmor
● kubearmor.service - KubeArmor
     Loaded: loaded (/lib/systemd/system/kubearmor.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-09-20 10:24:57 IST; 4min 53s ago
   Main PID: 594 (kubearmor)
      Tasks: 9 (limit: 2263)
     Memory: 116.1M
     CGroup: /system.slice/kubearmor.service
             └─594 /opt/kubearmor/kubearmor

Sep 20 10:25:05 ubuntu kubearmor[594]: 2023-09-20 10:25:05.403822        WARN        Error while looking for CRI socket file
Sep 20 10:25:05 ubuntu kubearmor[594]: 2023-09-20 10:25:05.404164        WARN        Failed to monitor containers:  is not a supported CRI socket.
Sep 20 10:25:05 ubuntu kubearmor[594]: 2023-09-20 10:25:05.404368        INFO        Using  for monitoring containers
Sep 20 10:25:06 ubuntu kubearmor[594]: 2023-09-20 10:25:06.404647        INFO        Started to monitor host security policies on gRPC
Sep 20 10:25:06 ubuntu kubearmor[594]: 2023-09-20 10:25:06.404695        INFO        Started to serve gRPC-based log feeds
Sep 20 10:25:06 ubuntu kubearmor[594]: 2023-09-20 10:25:06.404707        INFO        Initialized KubeArmor
Sep 20 10:25:06 ubuntu kubearmor[594]: 2023-09-20 10:25:06.404763        WARN        Policies dir not found for restoration
Sep 20 10:27:04 ubuntu kubearmor[594]: 2023-09-20 10:27:04.392948        INFO        Added a new client (6b6eea3a-c09f-4af1-b885-f1ad5fc37552, policy) for WatchAlerts
Sep 20 10:27:47 ubuntu kubearmor[594]: 2023-09-20 10:27:47.784275        INFO        Detected a Host Security Policy (added/pkg-mngr-exec)
Sep 20 10:27:47 ubuntu kubearmor[594]: 2023-09-20 10:27:47.894093        INFO        Updated 31 host security rules to the KubeArmor host profile in ubuntu
vboxuser@ubuntu:~$ karmor logs --gRPC=:32767&
[1] 1979
vboxuser@ubuntu:~$ Created a gRPC client (:32767)
Checked the liveness of the gRPC server
Started to watch alerts

vboxuser@ubuntu:~$ apt
== Alert / 2023-09-20 05:00:19.098045 ==
HostName: ubuntu
Type: MatchedHostPolicy
PolicyName: pkg-mngr-exec
Severity: 5
Message: Alert! Execution of package management process inside container is denied
Source: /usr/bin/bash
Resource: /usr/bin/apt
Operation: Process
Action: Block
Data: syscall=SYS_EXECVE
Enforcer: AppArmor
Result: Permission denied
ATags: [NIST NIST_800-53_CM-7(4) SI-4 process NIST_800-53_SI-4]
HostPID: 1986
HostPPID: 1943
PID: 1986
PPID: 1943
ParentProcessName: /usr/bin/bash
ProcessName: /usr/bin/apt
Tags: NIST,NIST_800-53_CM-7(4),SI-4,process,NIST_800-53_SI-4
UID: 1000
-bash: /usr/bin/apt: Permission denied

This is great.