Available bandwidth not maxed out on Windows

[ylangisc]

Using SSL/TLS based protocols there is a difference in maximum transfer speeds between macOS and Windows for large bandwidth connections. This seems to be related to IKVM used for the Windows version. While plain Java on both platforms is able to max out the bandwidth for a 1GBit/s connection, the IKVM based version is limited to ~430MBit/s.

Sample program to measure speed for TLS 1.2 connections:

package ch.iterate;

import javax.net.ssl.*;
import java.io.IOException;
import java.io.InputStream;
import java.net.*;
import java.security.SecureRandom;
import java.time.Duration;
import java.time.Instant;

public class Main {

    public static void main(String[] args) throws Exception {
        //System.setProperty("javax.net.debug", "ssl,handshake");

        final URL connect = new URL(args[0]);
        URLConnection urlConnection = connect.openConnection();
        HttpURLConnection http = (HttpURLConnection) urlConnection;
        if (http instanceof HttpsURLConnection) {
            HttpsURLConnection https = (HttpsURLConnection) http;
            https.setSSLSocketFactory(new TrustSslSocket());
            https.setHostnameVerifier((hostname, session) -> true);
        }
        http.setRequestMethod("GET");
        http.connect();
        if (http.getResponseCode() != HttpURLConnection.HTTP_OK) {
            return;
        }
        final long length = http.getHeaderFieldLong("Content-Length", -1);
        if (length == -1) {
            return;
        }
        Instant before = Instant.now();
        long read = 0;
        byte[] buffer = new byte[4 * 1024 * 1024];
        try (final InputStream stream = http.getInputStream()) {
            int tmp;
            while ((tmp = stream.read(buffer)) != -1 && read < length) {
                read += tmp;
            }
        }
        final Duration runtime = Duration.between(before, Instant.now());
        System.out.println(length / runtime.getSeconds());
    }

    final static class TrustAll implements X509TrustManager {
        @Override
        public void checkClientTrusted(java.security.cert.X509Certificate[] chain, String authType) {
        }

        @Override
        public void checkServerTrusted(java.security.cert.X509Certificate[] chain, String authType) {
        }

        @Override
        public java.security.cert.X509Certificate[] getAcceptedIssuers() {
            return new java.security.cert.X509Certificate[]{};
        }
    }

    final static class TrustSslSocket extends SSLSocketFactory {

        private final SSLContext context;
        private final SSLSocketFactory factory;

        public TrustSslSocket() throws Exception {
            // Default provider
            context = SSLContext.getInstance("TLSv1.2");
            context.init(null, new TrustManager[]{new TrustAll()}, new SecureRandom());
            factory = context.getSocketFactory();
        }

        @Override
        public Socket createSocket() throws IOException {
            return factory.createSocket();
        }

        @Override
        public Socket createSocket(final String host, final int port, final InetAddress clientHost, final int clientPort) throws IOException {
            return factory.createSocket(host, port, clientHost, clientPort);

        }

        @Override
        public Socket createSocket(final InetAddress host, final int port) throws IOException {
            return factory.createSocket(host, port);
        }

        @Override
        public Socket createSocket(final InetAddress host, final int port, final InetAddress localHost, final int localPort) throws IOException {

            return factory.createSocket(host, port, localHost, localPort);
        }

        @Override
        public Socket createSocket(final String host, final int port) throws IOException {
            return factory.createSocket(host, port);
        }

        @Override
        public Socket createSocket(final Socket socket, final String host, final int port, final boolean autoClose) throws IOException {
            return factory.createSocket(socket, host, port, autoClose);
        }

        @Override
        public String[] getDefaultCipherSuites() {
            return factory.getDefaultCipherSuites();
        }

        @Override
        public String[] getSupportedCipherSuites() {
            return factory.getSupportedCipherSuites();
        }
    }
}

Test results on Windows with a 1GBit/s connection between client and web server with TLS 1.2:	Scenario	Throughput
Eclipse Adoptium jdk-8.0.372.7	119304647 Bytes/s
Eclipse Adoptium jdk-8.0.372.7 with disabled hardware acceleration (-XX:+UnlockDiagnosticVMOptions -XX:-UseAES -XX:-UseAESIntrinsics)	107374182 Bytes/s
IKVM 8.0.312.7	53687091 Bytes/s
ikvmnet	? Bytes/s

It's interesting that the difference between IKVM and Java with disabled hardware acceleration is exactly factor 2. Looks like the AES implementation runs on 1 vs. 2 or 2 vs. 4 cores ....

[AliveDevil]

With -XX:-UseGHASHIntrinsics observing the same bandwidth as IKVM on JDK.

[MarekBabala]

Greetings, I just wanted share some information i have found out recently, and its the fact that there are multiple S3 Configuration "options" for AWS S3 CLI , basically to control the flow of the data from S3 Bucket to your workstation, from what I know they control, both AWS S3 CP command and the AWS S3api calls. From what I can see above the issue may be related to some Java issues, but overall , these "TEAKABLE" options would be nice to have in the CyberDuck Interface, because its really not one size fits all, and there is no "auto" feature thus far.

All of those features are described here:

https://awscli.amazonaws.com/v2/documentation/api/latest/topic/s3-config.html

and lots of these 'hidden' options are omitted on the AWS S3 CLI pages.

Thanks for taking a look at this and possible implementing some of these 'features' in the S3 Transfer tab.