allow docker host access to enclave ports under linux ?

[garyng2000]

What's your CLI version?

0.90.1

Description & steps to reproduce

I am using a kurtosis package at github.com/ethpandaops/ethereum-package

which maps host ports to docker container ones(created via kurtosis) but I cannot access it from the host. how should the kurtosis package to be configured in order to allow this ?

Desired behavior

should be able to access mapped port from host

What is the severity of this bug?

Critical; I am blocked and Kurtosis is unusable for me because of this bug.

What area of the product does this pertain to?

Other: anything not covered by the above

[tedim52]

Hey @garyng2000 ! How are you attempting to access the ports? eg. localhost:61234with 61234 being the port on the host mapped to the port of the service inside the enclave ?

[garyng2000]

curl -s http://localhost: as shown in 'kurtosis enclave inspect '

========================================== User Services ========================================== UUID Name Ports Status 9115b4d644b6 cl-1-lighthouse-geth http: 4000/tcp -> http://127.0.0.1:32877 RUNNING metrics: 5054/tcp -> http://127.0.0.1:32876 tcp-discovery: 9000/tcp -> 127.0.0.1:32875 udp-discovery: 9000/udp -> 127.0.0.1:32789 683d6f13726e cl-2-lighthouse-geth http: 4000/tcp -> http://127.0.0.1:32880 RUNNING metrics: 5054/tcp -> http://127.0.0.1:32879 tcp-discovery: 9000/tcp -> 127.0.0.1:32878 udp-discovery: 9000/udp -> 127.0.0.1:32790 f5803bd182e3 cl-3-teku-geth http: 4000/tcp -> http://127.0.0.1:32883 RUNNING metrics: 8008/tcp -> 127.0.0.1:32882 tcp-discovery: 9000/tcp -> 127.0.0.1:32881 udp-discovery: 9000/udp -> 127.0.0.1:32791 7691562c9fb5 dora http: 8080/tcp -> http://127.0.0.1:32886 RUNNING 4a24f863b367 el-1-geth-lighthouse engine-rpc: 8551/tcp -> 127.0.0.1:32862 RUNNING metrics: 9001/tcp -> 127.0.0.1:32861 rpc: 8545/tcp -> http://127.0.0.1:32864 tcp-discovery: 30303/tcp -> 127.0.0.1:32860 udp-discovery: 30303/udp -> 127.0.0.1:32786 ws: 8546/tcp -> 127.0.0.1:32863 e90e9bb1ee16 el-2-geth-lighthouse engine-rpc: 8551/tcp -> 127.0.0.1:32867 RUNNING metrics: 9001/tcp -> 127.0.0.1:32866 rpc: 8545/tcp -> http://127.0.0.1:32869 tcp-discovery: 30303/tcp -> 127.0.0.1:32865 udp-discovery: 30303/udp -> 127.0.0.1:32787 ws: 8546/tcp -> 127.0.0.1:32868 6c2021fe91ad el-3-geth-teku engine-rpc: 8551/tcp -> 127.0.0.1:32872 RUNNING metrics: 9001/tcp -> 127.0.0.1:32871 rpc: 8545/tcp -> http://127.0.0.1:32874 tcp-discovery: 30303/tcp -> 127.0.0.1:32870 udp-discovery: 30303/udp -> 127.0.0.1:32788 ws: 8546/tcp -> 127.0.0.1:32873 68d7758bdc9c validator-key-generation-cl-validator-keystore RUNNING 4be8bbb05868 vc-1-geth-lighthouse metrics: 8080/tcp -> http://127.0.0.1:32884 RUNNING 22175e303736 vc-2-geth-lighthouse metrics: 8080/tcp -> http://127.0.0.1:32885 RUNNING

[garyng2000]

so in this case curl -s http://localhost:32864. I get reset by peer error I can get into the the container shell and access it without issue

wondering if it has anything to do with this

host.docker.internal:host-gateway

which is needed for my other docker-compose setup

below is the exact input/output root@hqprdubada:/mnt/data3/chatgpt# curl -X POST -H "Content-Type: application/json" --data '{"method":"eth_blockNumber","params":[],"id":1,"jsonrpc":"2.0"}' http://127.0.0.1:32864 curl: (56) Recv failure: Connection reset by peer

[tedim52]

what are the network params you're running with? or are you using defaults? Also how are you seeing the host.docker.internal config/does it work if you unset it?

my hypothesis is that this setting host.docker.internal:host-gateway might make services in the enclave communicate on the host network which means when the el-1-geth-lighthouse attempts to connect with other peers on the IP address assigned to them on the enclave subnetwork, that IP address is invalid on the host network?

[garyng2000]

I am not using additional param, just straight setup from this(basically default)

https://geth.ethereum.org/docs/fundamentals/kurtosis

it seems that the 'dora' service can communicate with the geth container so it must be some docker level setup issue but that being wrapped inside kurtosis, I don't know where to start to trace

[tedim52]

Any chance you can join our discord? I'm happy to continue assisting you there! https://discord.com/invite/HUapYX9RvV

[garyng2000]

thanks for the help, I would try to use that channel later.

I may have find the reason as inspecting the geth container itself shows me this about network

"Networks": { "kt-private3151908net": { "IPAMConfig": { "IPv4Address": "172.16.0.13" }, "Links": null, "Aliases": [ "el-3-geth-teku" ], "MacAddress": "02:42:ac:10:00:0d", "NetworkID": "150bb87a432c106db2b39f86325828a0e210a8d264e137787858bc42d8f6664d", "EndpointID": "2951b8e68dbfbce91a7a8454bdda0bd8a3af13c46837b4696ce0b20beae6fa7b", "Gateway": "172.16.0.4", "IPAddress": "172.16.0.13", "IPPrefixLen": 22, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "DriverOpts": null, "DNSNames": [ "el-3-geth-teku--6c2021fe91ad46d5b8e12db3a8fc7c82", "el-3-geth-teku", "81f987de1343" ] } }

which may be conflicting with my hosting machine which is also using 172.16.0.x, so the question may then be anyway to configure the network setup in kurtosis for the 'private docker network' ? though just a WAG at the moment

[tedim52]

That makes sense. Background - Kurtosis creates a Docker bridge subnetwork for each enclave so that services within each enclave can only communicate with other services in the enclave. Any custom config that might alter how Docker networking works might affect this.

[garyng2000]

something doesn't make sense, in the created brdige(which seems to be what kurtosis is using)

br-b4e4c03a202a: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1440 inet 172.16.0.2 netmask 255.255.252.0 broadcast 172.16.3.255 inet6 fe80::42:53ff:fe3f:35cc prefixlen 64 scopeid 0x20 ether 02:42:53:3f:35:cc txqueuelen 0 (Ethernet) RX packets 215 bytes 26166 (26.1 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 51 bytes 6221 (6.2 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

so this is using a bridging network thus the host mapping probably is not working as my limited understanding of docker is it needs to be in 'host' network mode for the port mapping to work

[garyng2000]

ok, it seems that it is the package itself https://github.com/ethpandaops/ethereum-package/tree/main is having issue, the default quickstart package runs find as the proper network configurations are used for the various containers

need to bug the package maintainer :-)