search

3x3cut0r / docker

several docker containers
https://github.com/3x3cut0r/docker
23 stars 9 forks source link

tftpd-hpa docker swarm issue #2

Closed CMALG closed 2 years ago

CMALG commented 2 years ago

Hi first thanks for providing the container it's exactly what i was looking for :) But i have some strange issues with it.

If i run this i have 0 issues:

docker run -d \
    --name tftpd-hpa \
    -e TZ="Europe/Berlin" \
    -e CREATE=1 \
    -v /var/tftp:/tftpboot \
    -p 69:69/udp \
    3x3cut0r/tftpd-hpa:latest
INFO: /usr/sbin/in.tftpd --foreground --address 0.0.0.0:69 --user tftp  --create --port-range 4096:32760 --map-file /mapfile --secure --verbose --verbosity 3 /tftpboot
Aug 11 11:04:12 6f299c44355e syslog.info syslogd started: BusyBox v1.34.1
Aug 11 09:04:53 6f299c44355e daemon.info in.tftpd[63]: remap: input: test.txt
Aug 11 09:04:53 6f299c44355e daemon.info in.tftpd[63]: remap: rule 6: rewrite: /test.txt
Aug 11 09:04:53 6f299c44355e daemon.info in.tftpd[63]: remap: done
Aug 11 09:04:53 6f299c44355e daemon.notice in.tftpd[63]: WRQ from 192.168.3.222 filename test.txt remapped to /test.txt

But since i want to run it as a stack using you compose file as a base configuration:

version: '3'
services:
  tftpd-hpa:
    image: 3x3cut0r/tftpd-hpa
    environment:
        CREATE: 1
    volumes:
      - /var/tftp:/tftpboot
    ports:
      - 69:69/udp
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints: [node.role == worker]
INFO: /usr/sbin/in.tftpd --foreground --address 0.0.0.0:69 --user tftp  --create --port-range 4096:32760 --map-file /mapfile --secure --verbose --verbosity 3 /tftpboot
Aug 11 09:06:59 afa464d9a213 syslog.info syslogd started: BusyBox v1.34.1
Aug 11 09:07:38 afa464d9a213 daemon.info in.tftpd[67]: remap: input: test.txt
Aug 11 09:07:38 afa464d9a213 daemon.info in.tftpd[67]: remap: rule 6: rewrite: /test.txt
Aug 11 09:07:38 afa464d9a213 daemon.info in.tftpd[67]: remap: done
Aug 11 09:07:38 afa464d9a213 daemon.notice in.tftpd[67]: WRQ from 10.0.0.3 filename test.txt remapped to /test.txt
Aug 11 09:07:38 afa464d9a213 daemon.warn in.tftpd[67]: tftpd: write(ack): Operation not permitted

I get "Operation not permitted".

I see no reason why it should not work but it does not for some strange reason. so my question have you ever tried to run it in a stack on docker swarm ? If not you can close the issue it is a long shot because i'm completely out of ideas....

3x3cut0r commented 2 years ago

no i never run it in swarm mode. it was for me for some local stuff. but i checked if i could run it on a vps. but i have just a docker-rootless instance (still no swarm). there i could run it but only on a custom port 6969 (i think only ports >1024 are working without to add some capabilities).

here is my stack config:


services:
    tftp-hpa:
        container_name: tftp-hpa
        image: 3x3cut0r/tftpd-hpa
        restart: unless-stopped
        ports:
           - "6969:69/udp"
        networks:
            tftp-hpa:
                ipv4_address: 10.24.69.69
        dns:
            - 8.8.8.8
            - 8.8.4.4
            - 2001:4860:4860::8888
            - 2001:4860:4860::8844
        environment:
            TZ: "Europe/Berlin"
            CREATE: 1
        volumes:
            - tftp-hpa-data:/tftpboot

networks:
    tftp-hpa:
        name: tftp-hpa
        ipam:
            driver: default
            config:
                - subnet: "10.24.69.0/24"
                  gateway: 10.24.69.1

volumes:
    tftp-hpa-data:
        name: tftp-hpa-data

do you need to use port 69 for some reason? then i think you need to add some capabilities? see here: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities

try to add "--cap-add=ALL" to check if it is working. and then reduce it to the needed one. most likely "--cap-add=NET_BIND_SERVICE"

for a stack: https://docs.docker.com/compose/compose-file/compose-file-v3/#cap_add-cap_drop

cap_add:
  - ALL

i hope this can help

CMALG commented 2 years ago

Thank you very much for the ideas unfortunately same outcome can't write files as soon as i deploy in stack on the same volumes. Even with cap_add all it will not work. But since nobody has any issues it might have something to do with my raspberrys. Who knows.

3x3cut0r commented 2 years ago

After some googling i found the problem (and solution) of your issue here: https://www.linuxquestions.org/questions/linux-server-73/tftp-connection-problem-626077/

and after looking deeper into other tftp-hpa docker images i found that you need to load netfilter modules and add helpers to track connections on your host machine:

sudo modprobe nf_conntrack_tftp
sudo modprobe nf_nat_tftp

can u try this and let us know if it helps

CMALG commented 2 years ago

Thanks man but unfortunately still the same.

Checked that the modules are in fact activated on both manager and node.

ubuntu@Manager-1:~$ find /lib/modules/$(uname -r) | grep tftp
/lib/modules/5.15.0-1013-raspi/kernel/net/netfilter/nf_conntrack_tftp.ko
/lib/modules/5.15.0-1013-raspi/kernel/net/netfilter/nf_nat_tftp.ko

same error message

INFO: /usr/sbin/in.tftpd --foreground --address 0.0.0.0:69 --user tftp  --create --port-range 4096:32760 --map-file /mapfile --secure --verbose --verbosity 3 /tftpboot
Aug 16 11:03:31 2b74865b4caa daemon.info in.tftpd[2959]: remap: input: test.txt
Aug 16 10:28:17 2b74865b4caa syslog.info syslogd started: BusyBox v1.34.1
Aug 16 11:03:31 2b74865b4caa daemon.info in.tftpd[2959]: remap: rule 6: rewrite: /test.txt
Aug 16 11:03:31 2b74865b4caa daemon.info in.tftpd[2959]: remap: done
Aug 16 11:03:31 2b74865b4caa daemon.notice in.tftpd[2959]: WRQ from 10.0.0.3 filename test.txt remapped to /test.txt
Aug 16 11:03:31 2b74865b4caa daemon.warn in.tftpd[2959]: tftpd: write(ack): Operation not permitted
3x3cut0r commented 2 years ago

What you did with your find command is to check if the libraries are installed and not if there are loaded. please check it with

lsmod | grep ftp

the outcome should be something like that:

nf_nat_tftp            16384  0
nf_conntrack_tftp      20480  1 nf_nat_tftp
nf_nat                 57344  5 xt_nat,nf_nat_tftp,nft_chain_nat,iptable_nat,xt_MASQUERADE
nf_conntrack          176128  7 xt_conntrack,nf_nat,nf_conntrack_tftp,xt_nat,nf_nat_tftp,nf_conntrack_netlink,xt_MASQUERADE

as i said, activate them with:

sudo modprobe nf_conntrack_tftp
sudo modprobe nf_nat_tftp
CMALG commented 2 years ago

sorry should have been more clear, i checked if they are present and activated them.

ubuntu@Node-1:~$ lsmod | grep ftp
nf_nat_tftp            16384  0
nf_conntrack_tftp      20480  1 nf_nat_tftp
nf_nat                 49152  5 xt_nat,nf_nat_tftp,nft_chain_nat,xt_MASQUERADE,xt_REDIRECT
nf_conntrack          184320  10 xt_conntrack,nf_nat,nf_conntrack_tftp,xt_state,xt_nat,nf_nat_tftp,nf_conntrack_netlink,xt_MASQUERADE,ip_vs,xt_REDIRECT
3x3cut0r commented 2 years ago

Ok. Please don't feel sold for stupid, but i also just want to be sure.

So i will close the issue again. But feel free to open it again or contact me if you have any ideas.

CMALG commented 2 years ago

No no was my fault for not being clear what i did.

I did some more testing and to my surprise it seems like a docker bug maybe ?

This does start and work

docker service create \
  --mode global \
  --publish mode=host,target=69,published=69,protocol=udp \
  --name=tftp \
  -e TZ="Europe/Berlin" \
  -e CREATE=1 \
3x3cut0r/tftpd-hpa:latest

If you change publish mode to ingress the containers are stuck in "new" and will never start, waited hours....

docker service create \
  --mode global \
  --publish mode=ingress,target=69,published=69,protocol=udp \
  --name=tftphost \
  -e TZ="Europe/Berlin" \
  -e CREATE=1 \
3x3cut0r/tftpd-hpa:latest

For some reason if you run the container with an ingress overlay network of any kind it does not work.

Seems more of a docker issue, i close this if i ever get it working i will let you know :D