Install falco and sysdig on compute nodes

[martbhell]

This provides some useful tools like:

• csysdig (a top for containers)
• falco (a service that audits suspicious activity inside containers)

For now only in compute.yml (so not for ansible-pull).

Use CSCfi/ansible-falco fork. The old sysdig role we looked at in #191 has disappeared

Have tested that "journalctl -xefu falco" writes things to the journal when "event_generator -a all" is run from within this container:

singularity run docker://sysdig/falco-event-generator

[martbhell]

This has been running on one of our compute nodes in the test cluster for a week. The log file of that node on the admin node's central log is about 5x larger gzip compressed (from 100KiB to 500KiB). A lot of the messages are from falco (which are matching because it's complaining about ansible).

$ sudo zcat io1.log-20180411.gz |wc -l
22344
$ sudo zcat io1.log-20180411.gz |grep -c falco
18805

I'm thinking before this is merged we should add some rules to have falco not complain about ansible-pull.

https://github.com/draios/falco/wiki/Falco-Rules

[martbhell]

Would be sweet if we could install sysdig without kernel modules but https://github.com/draios/sysdig/wiki/eBPF-(beta) seems to be in very early stages and requires really new kernel >=4.14 for features I don't know are backported to RHEL7.

[rkdarst]

Can this old PR from 2018 be closed now?

I'm looking at old PRs from my organizations, and this is a very old PR, and I guess it can be closed now since probably there is another solution. But I can't do it myself.

[VilleS1]

Yeah lets close it.