Container network interfaces dependency loop

[deliciouslytyped]

To be able to view this problem through a more concrete lens, I have the following situation:

• I have a VPN service which requires internet access to download some configuration files (at least on the first start)
• I am running this in a container
• a container is not up until its services are up (?)
• (the problem is here!) the host does not enable the host side veth until the container is up, in a post-script, see: https://github.com/NixOS/nixpkgs/blob/c3b0e1268a3786b0ec00f1bd6accca5b1d57d550/nixos/modules/virtualisation/containers.nix#L198
• if the host side veth is not configured the container has no internet

Therefore, the vpn cannot download the configuration files until the vpn is up, contradiction.

I did a dirty hack of just dumping the postscript into the prescript so the interface is brought up properly on time, and this seems to work (did it in the wrong place, don't have a solution yet) - but is there a better solution? Will pre-activating the interface break anything? Is there some refactoring possible in the above linked implementation?

[deliciouslytyped]

Err, I wrote the issue but something might still be messed up on my part so I still need to check some things...

[deliciouslytyped]

So, for my specific case, systemd-nspawn in the container ExecStart is what provides the host veth, which means that the host veth needs to/can be be configured immediately after nspawn starts running, but before it finishes or times out. (I'm not sure how exactly it works but I'm assuming it stops blocking with respect to systemd service queueing when the contianer comes up somehow.)

[deliciouslytyped]

Apparently the systemd service is of notify type, which makes a lot of sense given the above; ; however, it is expected that the service sends a notification message via sd_notify(3) or an equivalent call when it has finished starting up. - infinisil mentioned something suggesting that execstart would block the service forever unless something else is happening.

[deliciouslytyped]

The code responsible for bringing up the host veth is currently in ExecStartPost and that's too late.

[deliciouslytyped]

So, I think interfaces should be brought up as soon as the device appears, since that is the logical dependency.

[stale[bot]]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[alex-fu27]

I can confirm this problem.

Take for example, the Gitlab module: gitlab-db-config.service needs to communicate with the Postgresql server, which is reachable over network, but network will not be set online until the service has finished.

[dantefromhell]

@deliciouslytyped Have you found a good solution for this?

From what I can see every service that requires network access to startup successfully (e.g. gitlab as described by @alex-fu27) is unsupported in nixos containers.

[deliciouslytyped]

I haven't touched this in 5 (wow) years, sorry. :(