Im trying to setup a environment where every container connection gets routed over the default gateway so i can setup firewall rules and isolate all containers to my liking on the main firewall.
I was able to setup dhcp using pipework which is already great so i can manage ip's etc on the firewall and do proper dns resolution without having to manually set overrides.
Pipework by default uses a macvlan in bridge mode which bypasses the default gateway for container to container communication.
So right now i can control traffic that goes out from one docker host to any other host on the network for example container1 from docker1 to container2 on docker2 host.
However container1 to container3 both on host docker1 are bypassing the gateway.
After some research online i found out that this is the intended behaviour of macvlan in bridge mode.
After changing mode bridge to mode vepa in /sbin/pipework the network should work how i intended it.
The problem is with vepa mode connections between containers on the same host arent working at all anymore.
If i do a tcpdump and ping from container1 to container3 i see arp requests that get no answer.
Everything right now is a virtual enviroment using proxmox as host with opnsense as firewall and debian10 as docker hosts, so there should be no hardware limitation.
I enabled hairpin mode on the proxmox bridge.
Anyone has a simmilar setup that is working or might be able to help me with that ?
I dont understand why docker after 6 years still does not support such basic network setups.
Im trying to setup a environment where every container connection gets routed over the default gateway so i can setup firewall rules and isolate all containers to my liking on the main firewall.
I was able to setup dhcp using pipework which is already great so i can manage ip's etc on the firewall and do proper dns resolution without having to manually set overrides.
Pipework by default uses a macvlan in bridge mode which bypasses the default gateway for container to container communication.
So right now i can control traffic that goes out from one docker host to any other host on the network for example container1 from docker1 to container2 on docker2 host.
However container1 to container3 both on host docker1 are bypassing the gateway.
After some research online i found out that this is the intended behaviour of macvlan in bridge mode.
After changing mode bridge to mode vepa in /sbin/pipework the network should work how i intended it.
The problem is with vepa mode connections between containers on the same host arent working at all anymore.
If i do a tcpdump and ping from container1 to container3 i see arp requests that get no answer.
Everything right now is a virtual enviroment using proxmox as host with opnsense as firewall and debian10 as docker hosts, so there should be no hardware limitation.
I enabled hairpin mode on the proxmox bridge.
Anyone has a simmilar setup that is working or might be able to help me with that ?
I dont understand why docker after 6 years still does not support such basic network setups.