Question about macvlan for multiple AD

killmasta93 commented 1 year ago

Hi I was testing out macvlan, on my linux host because i wanted to see if i can have 2 AD domains in 2 containers for dc1 its working great as its bridge, but when i try to add dc2 and create the macvlan it seems that the ports dont seem to get opened is there a limitation on the docker file?

These were the steps i took

created the docker network

sudo docker network create \
        --driver macvlan \
        --subnet 192.168.7.0/24 \
        --gateway 192.168.7.254 \
        --ip-range 192.168.7.100/29 \
        --opt parent=ens18 \
        macvlan_net

this is the docker compose of dc2


version: "3.1"

services:
  dc2:
    image: samba:ubuntu
    restart: unless-stopped
    build:
      context: .
      dockerfile: dockerfiles/ubuntu
      args:
        SMB_VERSION: "${SMB_VERSION:-latest}"
    container_name: dc2
    hostname: DC2
    privileged: true
    environment:
      REALM: DGS2.NET
      DOMAIN: DGS2
      ADMIN_PASS: Passw0rd
      DNS_FORWARDER: 8.8.8.8
    networks:
      - macvlan_net
    volumes:
      - dc2-local-samba:/usr/local/samba

networks:
 macvlan_net:
   name: macvlan_net

volumes:
  dc2-local-samba:

it seems that in the container it shows the ports running

thank you

diegogslomp commented 1 year ago

Hello

Check the docker-compose main file that creates a multiple DC environment, specifically at the BIND_NETWORK_INTERFACES: false option, you only bind interfaces in network host mode. Any trouble let me know.

killmasta93 commented 1 year ago

thanks for the reply, i was looking at the main file, but it shows dc1 dc2 with the same dgs,net lets say i want dgs.net and another domain dgs2.net

Thank you

diegogslomp commented 1 year ago

Then you need to create two DCs without relation between them, maybe something like this:

version: "3.8"

services:
  dc1:
    image: diegogslomp/samba-ad-dc
    restart: unless-stopped
    container_name: dc1
    hostname: DC1
    privileged: true
    environment:
      REALM: DGS.NET
      DOMAIN: DGS
      ADMIN_PASS: Passw0rd
      DNS_FORWARDER: 8.8.8.8
      BIND_NETWORK_INTERFACES: false
    dns_search: dgs.net
    dns:
      - 172.16.238.10
    extra_hosts:
      - "DC1 DC1.dgs.net:172.16.238.10"
      - "DC2 DC2.dgs2.net:172.16.238.20"
    command: bash -c "samba-domain-provision && samba -F"
    volumes:
      - dc1-local-samba:/usr/local/samba
    networks:
      static-network:
        ipv4_address: 172.16.238.10

  dc2:
    image: diegogslomp/samba-ad-dc
    restart: unless-stopped
    container_name: dc2
    hostname: DC2
    privileged: true
    environment:
      REALM: DGS2.NET
      DOMAIN: DGS2
      ADMIN_PASS: Passw0rd
      DNS_FORWARDER: 8.8.8.8
      BIND_NETWORK_INTERFACES: false
    dns_search: dgs2.net
    dns:
      - 172.16.238.20
    extra_hosts:
      - "DC1 DC1.dgs.net:172.16.238.10"
      - "DC2 DC2.dgs2.net:172.16.238.20"
    command: bash -c "samba-domain-provision && samba -F"
    volumes:
      - dc2-local-samba:/usr/local/samba
    networks:
      static-network:
        ipv4_address: 172.16.238.20

volumes:
  dc1-local-samba:
  dc2-local-samba:

networks:
  static-network:
    ipam:
      config:
        - subnet: 172.16.238.0/24

killmasta93 commented 1 year ago

thank you so much for the reply, the issue is that i need to bind the network such as networkmode to host so when my windows connect to the domain they can connect with no issue showing others IPs

So this is my DC1 and is working perfectly


version: "3.1"

services:
  dc1:
    image: samba:ubuntu
    restart: unless-stopped
    network_mode: host
    build:
      context: .
      dockerfile: dockerfiles/ubuntu
      args:
        SMB_VERSION: "${SMB_VERSION:-latest}"
    container_name: dc1
    hostname: DC1
    privileged: true
    environment:
      REALM: DGS.NET
      DOMAIN: DGS
      ADMIN_PASS: Passw0rd
      DNS_FORWARDER: 8.8.8.8
    volumes:
      - dc1-local-samba:/usr/local/samba

volumes:
  dc1-local-samba:
    driver: local-persist
    driver_opts:
     mountpoint: /data2

and this is my DC2 which loads buts when i try to nmap the ports its shows that its closed not sure why


version: "3.1"

services:
  dc2:
    image: samba:ubuntu
    restart: unless-stopped
#    network_mode: host
    build:
      context: .
      dockerfile: dockerfiles/ubuntu
      args:
        SMB_VERSION: "${SMB_VERSION:-latest}"
    container_name: dc2
    hostname: DC2
    privileged: true
    environment:
      REALM: DGS2.NET
      DOMAIN: DGS2
      ADMIN_PASS: Passw0rd
      DNS_FORWARDER: 8.8.8.8
      BIND_NETWORK_INTERFACES: true
    networks:
      - macvlan_net
    volumes:
      - dc2-local-samba:/usr/local/samba

networks:
 macvlan_net:
   name: macvlan_net

volumes:
  dc2-local-samba:

docker inspect macvlan_net
[
    {
        "Name": "macvlan_net",
        "Id": "b6cedf582369a8caeeab5cff129d26c8f442006862e637ca79e2c63f657a7fef",
        "Created": "2023-06-11T12:39:21.641386753-05:00",
        "Scope": "local",
        "Driver": "macvlan",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "192.168.7.0/24",
                    "IPRange": "192.168.7.100/29",
                    "Gateway": "192.168.7.254"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "e9da77b07c79c95606d21ba84b41825c1c9d291a18cdc37842f218e4c4b645c7": {
                "Name": "dc2",
                "EndpointID": "5b22dc8903dc74c0341f07de338ca82444117c34f013ae17753ee27bb490fc6f",
                "MacAddress": "02:42:c0:a8:07:60",
                "IPv4Address": "192.168.7.96/24",
                "IPv6Address": ""
            }
        },
        "Options": {
            "parent": "ens18"
        },
        "Labels": {}
    }
]

on the container of dc2 i can ping the gateway 192.168.7.254 and the windows server 192.168.7.105 no issue and vise versa


 nmap 192.168.7.96 -p 53
Starting Nmap 7.80 ( https://nmap.org ) at 2023-06-12 10:53 -05
Nmap scan report for 192.168.7.96
Host is up (0.00012s latency).

PORT   STATE  SERVICE
53/tcp closed domain
MAC Address: 02:42:C0:A8:07:60 (Unknown)

Nmap done: 1 IP address (1 host up) scanned in 0.32 seconds

Thank you

diegogslomp commented 1 year ago

Hello

When you use network mode, you expose the container ports to the outside of the docker, for example, the dns 53 port of your dc1 can be accessed from the outside, as if the dns server was installed on your host. Since dc2 uses the same ports as dc1, you can only expose one dns service on port 53, for example, to expose another dns 53 you need to shutdown one and start up another, or don't use the network mode and access then only inside docker, the way I use to test multiple dcs.

killmasta93 commented 1 year ago

Thank you so much for the reply, so i tried with your setup of docker compose, and im getting this error when try to connect to the domain from neither domains

my linux host ip

i tried nmap the linux host to see if the port is opened and shows closed


nmap 192.168.7.189 -p 88
Starting Nmap 7.80 ( https://nmap.org ) at 2023-06-12 22:15 -05
Nmap scan report for 192.168.7.189
Host is up (0.028s latency).

PORT   STATE  SERVICE
88/tcp closed kerberos-sec

Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds

Thank you

diegogslomp commented 1 year ago

This is the correct docker compose using network host mode:

version: "3.1"

services:
  dc1:
    image: diegogslomp/samba-ad-dc
    restart: unless-stopped
    network_mode: host
    container_name: dc1
    hostname: DC1
    privileged: true
    environment:
      REALM: DGS.NET
      DOMAIN: DGS
      ADMIN_PASS: Passw0rd
      DNS_FORWARDER: 8.8.8.8
    volumes:
      - dc1-local-samba:/usr/local/samba

volumes:
  dc1-local-samba:

Change your host /etc/resolv.conf updating dns server and nameserver:

search dgs.net
nameserver 192.168.7.189

Add the host to /etc/hosts:

127.0.0.1     localhost
192.168.7.189     DC1.dgs.net     DC1

More info configuring a domain controller here. Any trouble let me know.

killmasta93 commented 1 year ago

Thanks for the reply, allright so looking at it again it seems that i need network mode for both of the docker containers something like this

would your container be compatible something like that? as one container in ens18 network mode host and another ens19 network mode host

diegogslomp commented 1 year ago

Two interfaces, nice, don't know if there is a way to --network host each docker to each interface, I'll take a look

killmasta93 commented 1 year ago

Thank you yeah im trying to look though the docs of docker, but cant seem to find anything, it be very interesting to have varios AD in the same host with multiple NICS

diegogslomp commented 1 year ago

Did you try to create two macvlan networks, each one for each interface, and docker run them? If one works properly the other should work too, but should be separated networks, each one for each interface, I'm guessing, this macvlan option is new for me but make sense

killmasta93 commented 1 year ago

i did a macvlan on the same ens18 network and gave me another IP for that docker, but on my first post the ports did not seem to show open, which is odd because i tried it with another docker container wordpress and works fine

diegogslomp commented 1 year ago

Please send the wordpress conf to comparison, if it not use network host mode, the ports should be exposed manually in the compose file, or there is something with interface binding in smb.conf

killmasta93 commented 1 year ago

sure this is the wordpress

version: '3.9'

services:
  # Database
  db3:
    image: mysql:8.0
    volumes:
      - /wordpress3/db_data:/var/lib/mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: mypass
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: mypass
    networks:
      - macvlan_net
  # phpmyadmin
  phpmyadmin3:
    depends_on:
      - db3
    image: phpmyadmin/phpmyadmin
    restart: always
    ports:
     - "8085:80"
    environment:
      PMA_HOST: db3
      MYSQL_ROOT_PASSWORD: mypass
    networks:
      - macvlan_net
  # Wordpress
  wordpress3:
    depends_on:
      - db3
    image: wordpress:latest
    ports:
     - "8086:80"
    restart: always
    volumes:
      - /wordpress3/html:/var/www/html
    environment:
      WORDPRESS_DB_HOST: 192.168.7.96:3306
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: mypass
    networks:
      - macvlan_net

networks:
 macvlan_net:
   name: macvlan_net
volumes:
  db_data:
  html:

and created the network


sudo docker network create \
        --driver macvlan \
        --subnet 192.168.7.0/24 \
        --gateway 192.168.7.254 \
        --ip-range 192.168.7.100/29 \
        --opt parent=ens18 \
        macvlan_net

diegogslomp commented 1 year ago

So, this is the last conf with ports exposed and macvlan network already created, I think the interfaces should be bind like in network host mode, please check if you nmap the ports:

version: "3.1"

services:
  dc1:
    image: diegogslomp/samba-ad-dc
    restart: unless-stopped
    networks:
      - macvlan_net
    container_name: dc1
    hostname: DC1
    privileged: true
    environment:
      REALM: DGS.NET
      DOMAIN: DGS
      ADMIN_PASS: Passw0rd
      DNS_FORWARDER: 8.8.8.8
    ports:
      - "53:53"
      - "445:445"
    volumes:
      - dc1-local-samba:/usr/local/samba

volumes:
  dc1-local-samba:

networks:
  macvlan_net:
    name: macvlan_net

killmasta93 commented 1 year ago

thanks for the reply, i tried but im getting this issue

[+] Running 0/1
 ⠦ Container dc1  Starting                                                                                                                                                                                                            1.6s
Error response from daemon: driver failed programming external connectivity on endpoint dc1 (2177735c7998d66fe66a18a6eeb9e2ff10faf65e39e181b7c069199390d9d75a): Error starting userland proxy: listen tcp4 0.0.0.0:53: bind: address already in use

this are the ports currently on the linux host

systemd-r  843 systemd-resolve   13u  IPv4  22187      0t0  TCP 127.0.0.53:53 (LISTEN)
python3    934            odoo    4u  IPv4  35002      0t0  TCP *:8069 (LISTEN)
sshd      1107            root    3u  IPv4  25312      0t0  TCP *:22 (LISTEN)
sshd      1107            root    4u  IPv6  24505      0t0  TCP *:22 (LISTEN)
nginx     1207            root    8u  IPv4  26963      0t0  TCP *:443 (LISTEN)
nginx     1208        www-data    8u  IPv4  26963      0t0  TCP *:443 (LISTEN)
nginx     1209        www-data    8u  IPv4  26963      0t0  TCP *:443 (LISTEN)
postgres  1257        postgres    3u  IPv6  26474      0t0  TCP [::1]:5432 (LISTEN)
postgres  1257        postgres    4u  IPv4  26475      0t0  TCP 127.0.0.1:5432 (LISTEN)
docker-pr 2262            root    4u  IPv4  33319      0t0  TCP *:8085 (LISTEN)
docker-pr 2268            root    4u  IPv4  32566      0t0  TCP *:8083 (LISTEN)
docker-pr 2283            root    4u  IPv4  32565      0t0  TCP *:9000 (LISTEN)
docker-pr 2302            root    4u  IPv6  33322      0t0  TCP *:8083 (LISTEN)
docker-pr 2316            root    4u  IPv6  33353      0t0  TCP *:9000 (LISTEN)
docker-pr 2326            root    4u  IPv4  32591      0t0  TCP *:8084 (LISTEN)
docker-pr 2334            root    4u  IPv6  32596      0t0  TCP *:8085 (LISTEN)
docker-pr 2343            root    4u  IPv6  33374      0t0  TCP *:8084 (LISTEN)
docker-pr 2358            root    4u  IPv4  33438      0t0  TCP *:8000 (LISTEN)
docker-pr 2365            root    4u  IPv6  33445      0t0  TCP *:8000 (LISTEN)
docker-pr 2375            root    4u  IPv4  33448      0t0  TCP *:8086 (LISTEN)
docker-pr 2383            root    4u  IPv6  33451      0t0  TCP *:8086 (LISTEN)

i did the following to create the macvlan

sudo docker network create \
        --driver macvlan \
        --subnet 192.168.8.0/24 \
        --gateway 192.168.8.254 \
        --ip-range 192.168.8.100/29 \
        --opt parent=ens19 \
        macvlan_net

diegogslomp commented 1 year ago

You need to stop this systemd-resolve running at the 53, since the dns server will be delivered by the docker dc

killmasta93 commented 1 year ago

Thank you so much, that did the trick, im going to keep testing out and postback, the only issue is that now the linux host cannot ping google because of the DNS was turn off :(

diegogslomp commented 1 year ago

Use your dc dns changing the /etc/resolv.conf:

search dgs.net
nameserver 192.168.7.189

Add the host to /etc/hosts:

127.0.0.1     localhost
192.168.7.189     DC1.dgs.net     DC1

killmasta93 commented 1 year ago

yes amazing got it working, i did have to run a few other commands to work because running it on a VLAN


sudo ip link add macvlan_int link ens19 type macvlan mode bridge
sudo ip address add 192.168.8.220/32 dev macvlan_int
sudo ip link set macvlan_int up
sudo ip route add 192.168.8.96/32 dev macvlan_int

diegogslomp / samba-ad-dc

Question about macvlan for multiple AD #14