Dempsy / dempsy

Distributed Elastic Message Processing System
Apache License 2.0
195 stars 21 forks source link

TcpUtils.java only uses the first found non-local host address #89

Open dmsquires opened 6 years ago

dmsquires commented 6 years ago

When playing with multiple network gateways this becomes a problem with registering incorrect IP's in zookeeper. (eg can become non-trivial when using multiple gateways with Docker). The default behavior on CentOS with eth0 and docker0 in this case to pick up the docker0 gateway, which may not be what is desired.

public class TcpUtils {

    public static InetAddress getFirstNonLocalhostInetAddress() throws SocketException {
        final List<InetAddress> addrs = getAllInetAddress();
        return addrs.stream()
                .filter(i -> !i.isLoopbackAddress() && i instanceof Inet4Address)
                .findFirst()
                .orElseThrow(() -> new DempsyException("There are no non-local network interfaces among " + addrs));
    }

    public static List<InetAddress> getAllInetAddress() throws SocketException {
        final List<InetAddress> ret = new ArrayList<>();
        final Enumeration<NetworkInterface> netInterfaces = NetworkInterface.getNetworkInterfaces();
        while (netInterfaces.hasMoreElements()) {
            final NetworkInterface networkInterface = netInterfaces.nextElement();
            for (final Enumeration<InetAddress> loopInetAddress = networkInterface.getInetAddresses(); loopInetAddress.hasMoreElements();) {
                final InetAddress tempInetAddress = loopInetAddress.nextElement();
                ret.add(tempInetAddress);
            }
        }
        return ret;
    }

}
jimfcarroll commented 6 years ago

Yes. This is definitely a problem in Docker. I've fixed this in other systems multiple ways (e.g. a standalone REST service with a well known location that tells the process that hits it it's external IP, using DC/OS VIPs, docker-compose overlay networks). This is compounded by the problem that server ports in Dempsy are ephemeral so if you're not running with a "host" network it wont have the correct port either.

Other than making the functionality pluggable/customizable I'm not sure what to do. I'm open to suggestions. Let me know what you think.

dmsquires commented 6 years ago

This use case is using host mode, so we make sure that eth0 is the first network gateway listed by brute force at the OS level. Without going into an overly robust networking solution, for this use case simply passing in the network bridge name would work.

jimfcarroll commented 6 years ago

Okay. I'll make the network interface it selects configurable by name. I can have a change up probably later today. It will be on 0.9.1-SNAPSHOT.

I'll write tests for it but let me know if it works for you and I'll cut a release with the change. You'll be able to supply the setting in the Infrastructure configuration (e.g. https://github.com/Dempsy/dempsy/blob/0.9/dempsy-framework.impl/src/test/resources/td/node.xml#L20 ).

As an alternative, if you're interested, the 0.9 codebase has the ability to customize the IP address resolution with your own code. See https://github.com/Dempsy/dempsy/blob/0.9/dempsy-framework.core/src/main/java/net/dempsy/transport/tcp/AbstractTcpReceiver.java#L37 . It you're configuring programatically you can just call that on the NioReceiver, otherwise this can be set using Spring on the NioReceiver (e.g. it would be set as a property here https://github.com/Dempsy/dempsy/blob/0.9/dempsy-framework.impl/src/test/resources/td/transport-nio.xml#L10 ). This was added to support Docker.

I'm not sure if you noticed but the documentation is out of date. I've been working on updating it slowly but it's currently referencing 0.8.x and 0.9 was a significant refactor.

jimfcarroll commented 6 years ago

See #90 Let me know if that works for you