Unable to complete the packet transfer

anarumanchi commented 3 years ago

Hi, Great work on the project. Really useful stuff. However, I seem to be running into a problem and would be glad if anyone could help me solve the issue. I followed the setup and installation as mentioned on the docs at github pages, but despite my setup being complete, and getting the routers up, I am facing the following error:

Attaching to dane_daemon_1, dane_router-20ms-50Mbit_1, dane_router-200ms-5Mbit_1, dane_client-20ms-50Mbit-browsing_1, dane_client-20ms-50Mbit-streaming_1, dane_client-200ms-5Mbit-streaming_1, dane_client-200ms-5Mbit-browsing_1 daemon_1 | 11:25:17 [INFO] Listening for docker startup events. Will stop listening after 400 seconds. daemon_1 | 11:25:20 [INFO] [+] Setting up router dane_router-20ms-50Mbit_1 daemon_1 | 11:25:33 [ERROR] Uncaught exception daemon_1 | Traceback (most recent call last): daemon_1 | File "/scripts/daemon.py", line 390, in daemon_1 | routers, clients = listen_for_container_startup(timeout=400) daemon_1 | File "/scripts/daemon.py", line 328, in listen_for_container_startup daemon_1 | setup_router(router) daemon_1 | File "/scripts/daemon.py", line 92, in setup_router daemon_1 | raise Exception(f'Network configuration failed for router {router.name}.\n{output}') daemon_1 | Exception: Network configuration failed for router dane_router-20ms-50Mbit_1. daemon_1 | b"bc: bad expression at 'packets transmitted, 0 packets received, 100% packet loss'\nsh: invalid number ''\n" dane_daemon_1 exited with code 1

I am running the docker and instances on a windows machine, but have also tried to get it work with a Mac system but keep running into the same error. Thanks

parkeraddison commented 3 years ago

It looks like your router containers aren't connecting to the internet.

The main problem is occurring at

daemon_1 | b"bc: bad expression at 'packets transmitted, 0 packets received, 100% packet loss'\nsh: invalid number ''\n"

prompted by https://github.com/dane-tool/dane/blob/main/scripts/router/network-setup.sh#L55 because the ping test in the router never returned any values.

Hopefully the following can help you figure out why that's the case! Good luck.

How to debug

The most helpful way to debug is to edit your /built/docker-compose.yml to comment out services > daemon > command: ... so that the daemon script doesn't automatically run, like so:

# File: built/docker-compose.yml

networks:
  ...
services:
  client-20ms-50Mbit-none:
    ...
  daemon:
    # command: python /scripts/daemon.py  <<< Comment out the command!
    environment:
      DOCKER_HOST: tcp://host.docker.internal:2375
    image: parkeraddison/dane-daemon
    labels:
      com.dane.type: daemon
    volumes:
    - ../scripts/:/scripts/
  router-20ms-50Mbit:
    ...
version: '3.9'

then you can launch the services from your modified compose file by using

make raw

Finally, open up a new terminal and exec into your router

docker exec -it dane_router-20ms-50Mbit_1 sh

Are your containers connected to the internet at all?

First step would probably be to see if the router is connected to the internet before we mess with the networking at all. Within the router terminal try

#@router
ping 8.8.8.8

If you aren't getting any packets in response, check to see if any container that you start (e.g. docker run -it --rm alpine sh) can connect to the internet. If not, then the problem is with docker on your system(s). If it's just the DANE containers that don't connect, let me know. You can also try execing into the daemon and client containers to see if they can connect to the internet.

Is the issue with the gateway?

If the router was able to ping successfully, then I guess the issue lies later on.

Still within the router terminal, try the following commands (basically running parts of network-setup.sh by hand to see where the internet breaks)

#@router
service_name=router-20ms-50Mbit
latency=20ms
bandwidth=50Mbit
router_ip_ext=$(getent hosts "$service_name".default | cut -d ' ' -f1)
echo $router_ip_ext

then

#@router
iface_ext=$(ip route | grep "$router_ip_ext" | cut -d ' ' -f3)
iface_int=$(echo "$iface_ext" | tr 01 10)
gateway_ip_ext=$(echo "$router_ip_ext" | awk -F '.' '{print $1"."$2"."$3".1"}')
echo $gateway_ip_ext

finally

#@router
ip route replace default via "$gateway_ip_ext" dev "$iface_ext"
ping 8.8.8.8

Is the issue with IP tables?

If that worked, try

#@router
iptables -t nat -A POSTROUTING -o "$iface_ext" -j MASQUERADE
ping 8.8.8.8

anarumanchi commented 3 years ago

Thanks @parkeraddison. I was able to resolve the problem. Closing the issue.

dane-tool / dane