IPs of nested containers are unreachable

[a-jumani]

1. When I have just one mahimahi container (e.g. by just using mm-delay 20), I am assigned inet 100.64.0.2 and P-t-p 100.64.0.1. From some other shell (i.e. outside the container), I can ping 100.64.0.2 (which incurs an RTT of 40ms) and 100.64.0.1.

2. When I have 2 mahimahi containers chained together (e.g. mm-delay 20 mm-loss downlink 0.1), I am assigned inet 100.64.0.4 and P-t-p 100.64.0.3 within the inner container. From some other shell (i.e. outside both the containers), I cannot ping any of 100.64.0.3/4 but I can still ping 100.64.0.2 (i.e the outermost container - mm-delay 20) and 100.64.0.1.

Both setups shows $MAHIMAHI_BASE is set to 100.64.0.1.

Problem: I want to place a webserver inside mahimahi containers. If I just use 1 container, it is reachable. If I chain 2 containers together and start the server within the innermost container, it becomes unreachable.

Is there a fix or workaround that will solve my problem?

[keithw]

Each container is inside a NAT relative to its parent, so you will have the same problems you normally do when running a server inside a NAT. The easiest approach is to run the server outside the containers, and run the client inside the innermost one (opening a connection to $MAHIMAHI_BASE). If this doesn't work for you, can you tell us more about your use case? Thanks, Keith

[a-jumani]

I want to run the server inside a nested container (with some fixed delay and a mahimahi trace to be emulated) while the client is a mobile device. I can always simulate the delay at the server side using time.sleep(.) function instead.

Is there a way to do port forwarding within the containers? Or will it be introduced anytime in the future?

Thank you so much for a quick response.

[keithw]

You can set up port forwarding yourself with iptables, but we don't (currently) have code to do it automatically.

[a-jumani]

Great. I'll try that.

Edit: the commands described here worked for me to reach a server running inside a nested Mahimahi (mm) shell.

Note: it requires knowing what would be the inet address on the ingress interface (can be checked using "ifconfig"). E.g. the outermost mm shell will have inet address most probably 100.64.0.2 (so port forward to this one from main, non-mm shell). If there is another mm shell inside the outermost shell, it'll most probably have inet address 100.64.0.4 (so port forward to this one from outermost mm shell).

[wesmwoo]

Hi @a-jumani, I'm looking to set up a server inside a mm shell (mm-webreplay) as well. I don't have a ton of experience with iptables and ip routing -- would it be possible for you to share the list of commands you used to configure the forwarding?