OpenMods / OpenData

MIT License
32 stars 16 forks source link

Significant additional load time #40

Open xJon opened 6 months ago

xJon commented 6 months ago

OpenEye can add around a minute of extra load time to the client, depending on the setup. Quoted from @makamys:

it seems to be looping over all my network interfaces to gather their ip addresses so it can omit them from the submitted data, but it's implemented in a really inefficient way on the main thread

Due to this I'll stop using it in my modpacks, but it would be nice if it ever got addressed!

makamys commented 6 months ago

To add some more details, openeye.logic.Sanitizers#addLocalAddresses is the problematic method. It calls InetAddress#getHostName for all interfaces, which can take a very long time. I'm using Windows 10, and discovered the issue in OpenEye-1.7.10-0.8.

I wrote a small test program that gathers all addresses using the same method. It can take anywhere between 30 and 60 seconds to execute on my machine, and parallelizing it only reduces this by ~10 seconds. So probably a larger refactor would be needed that makes the entire process asynchronous.

Test program ```java import java.net.InetAddress; import java.net.NetworkInterface; import java.util.Arrays; import java.util.Collections; import java.util.Enumeration; import java.util.HashSet; import java.util.List; import java.util.Set; public class OpenEyeTest { public static void main(String[] args) { boolean verbose = Arrays.asList(args).contains("-v"); System.out.println("Enumerating interfaces... (Serial)"); long t0 = System.nanoTime(); addLocalAddresses(false, verbose); long t1 = System.nanoTime(); System.out.println("Serial enumeration took " + ((t1-t0) / 1_000_000_000.0) + "s"); System.out.println(); System.out.println("Enumerating interfaces... (Parallel)"); addLocalAddresses(true, false); long t2 = System.nanoTime(); System.out.println("Parallel enumeration took " + ((t2-t1) / 1_000_000_000.0) + "s"); } private static void addLocalAddresses(boolean parallel, boolean verbose) { Set ips = new HashSet<>(); Set hosts = new HashSet<>(); try { List itfs = Collections.list(NetworkInterface.getNetworkInterfaces()); if(parallel) { itfs.parallelStream().forEach(intf -> { Enumeration addresses = intf.getInetAddresses(); while(addresses.hasMoreElements()) { InetAddress address = (InetAddress)addresses.nextElement(); if (address != null) { ips.add(address.getHostAddress()); String host = address.getHostName(); if (!ips.contains(host)) { hosts.add(host); } } } }); } else { for(NetworkInterface intf : itfs) { if(verbose) { System.out.println("Interface: " + intf); } Enumeration addresses = intf.getInetAddresses(); while(addresses.hasMoreElements()) { InetAddress address = (InetAddress)addresses.nextElement(); if (address != null) { long t0 = System.nanoTime(); ips.add(address.getHostAddress()); String host = address.getHostName(); long t1 = System.nanoTime(); if(verbose) { System.out.println(" address: " + address); System.out.println(" host: " + host); System.out.println(" enumeration time: " + ((t1-t0)/1_000_000_000.0) + "s"); } if (!ips.contains(host)) { hosts.add(host); } } } } } } catch (Throwable var7) { System.err.println("Failed to get local IP adresses for sanitization"); var7.printStackTrace(); } } } ```