search

daeuniverse / dae

eBPF-based Linux high-performance transparent proxy solution.
GNU Affero General Public License v3.0
2.63k stars 165 forks source link

[Enhancement] Work with systemd-nspawn #471

Closed KiriKira closed 3 months ago

KiriKira commented 3 months ago

Improvement Suggestion

Try to run dae in systemd-nspawn container, the following two problems were encountered:

  1. rlimit image dae seems to force rlimit to be reset before running, even if LimitMEMLOCK=infinity has been set in systemd image

  2. bindmount /sys I tried to bind mount /sys to nspawn following docker's usage, but this causes nspawn to fail to start (since the cgroup is also in /sys) So I tried just mount /sys/fs/bpf, I don't know if that's enough because problem 1 blocked me. If you know what to do, let me know.

Potential Benefits

Provide a new option for running dae in containers, which will be much more 清真 than docker

dae-prow[bot] commented 3 months ago

Thanks for opening this issue!

mzz2017 commented 3 months ago

Sorry and this may not be something that dae can solve. These requirements are necessary for dae. Therefore, how to use dae through systemd-nspawn needs everyone to explore. Everyone is welcome to update progress and discuss here.

KiriKira commented 3 months ago

Hi @mzz2017 , in my understanding, for problem 1, it should be that dae fails in its attempt to remove Memlock and refuses to run, but systemd via LimitMEMLOCK=infinity should be enough, right? Is it possible to just add a judgment to skip this action?

mzz2017 commented 3 months ago

It is requiresite for ebpf program. You can remove the related code and compile it and try again.

KiriKira commented 3 months ago

Problem resolved by the following nspawn config:

# /etc/systemd/nspawn/ASA.nspawn

[Exec]
PrivateUsers=no
#LimitMEMLOCK=infinity
#LimitNOFILE=infinity
SystemCallFilter=@privileged

[Files]
Bind=/sys/fs/bpf

The key point is SystemCallFilter=@privileged and Bind=/sys/fs/bpf, which allow dae to run in nspawn container.

If anyone is interested in more, feel free to read: https://kirikira.moe/post/49/