BindException

[kprabha7]

Jetty version 9.3.0.v20150612.jar

Java version/vendor (use: java -version) >=8

OS type/version Android 9

Description I am trying to run a websocket client (org.eclipse.jetty.websocket.client.WebSocketClient) on Android platform as a background service. When the service is killed (for reliability testing) and restarted, I am getting bind exception. The exception is as follows:

06-02 10:07:43.586 5393 5493 E : Exception: java.net.BindException: Address already in use

This exception is thrown asynchronously usuing OnWebSocketError callback. So, I am unable to do proper retry or clean up and retry.

I understand the port is not released by previous process of service/Android run time/some native layer (guessing). Please suggest a way to perform proper cleanup of underlying socket so that port becomes available during restore of service.

[joakime]

WebSocketClient does not bind to a port.

This error is not from websocket client. Also, upgrade your version of WebSocketClient, as 9.3.x is deprecated / end of life. Consider using something in the 9.4.x series.

[kprabha7]

@joakime Thanks for the response

Do you mean client is unable to connect to the server?

Is this specific issue fixed in 9.4.x?

what additional information is required?

[joakime]

There's no issue for Jetty here.

You only get that error (java.net.BindException: Address already in use) if a Server is attempting to bind to a specific port to start listening for incoming connection attempts.

WebSocketClient is neither a server, nor does ever have a need to bind to a port, it cannot produce that error, there is no action that WebSocketClient takes that can trigger it.

Upgrading is recommended, as old, deprecated, end of life, versions are no longer supported.

[kprabha7]

Upgrading to version 9.4.40 didn't improve or fix this issue. And, during client object initialisation, I am doing the following to make sure websocket communication takes place on specific port of the host:

client.setBindAddress(new InetSocketAddress(host, port));

That's why my suspect was socket binding to port error. I was thinking it could be either because of TCP TIME_WAIT or JVM taking longer time to release resource (InetAddress in this case). What else could be the root cause?

Waiting for some time before retrying websocket client initialisation did not solve this issue.

is this solved in any particular version?

I am sure it is websocket client because I am instantiating org.eclipse.jetty.websocket.client.WebSocketClient

[joakime]

The OS plays an important part in this as well. It is the one ultimately in charge of freeing up the network stack for other processes to use potentially similar configurations.

Also, why are you binding to a specific address on Android?

On a typical android mobile device, there's no stable interfaces to bind to. As your device will hop from tower to tower, mobile to wifi, wifi to mobile, etc, resulting in new network interfaces constantly.

Don't bind a client address. Instead, subscribe to the android network services to know when your network interface changes, so that you know when you need to reconnect due to a network interface change.

[kprabha7]

My suspicion was some what similar since core networking must happen in native layers and ultimately handed over to kernel. There are several other layers before it reaches to Android application and JVM resources deallocation is somewhat unpredictable to the best of my knowledge.

Let me give you some more context.

I have access to Android platform sources for our device and the interfaces are created with VLAN IDs and statically configured during boot using native service. So, I can be 100% sure that it will not change dynamically anytime.

During the initial start up of service (triggered by BOOT_COMPLETED) there is 100% success in initaliazation and connection to server. But when service is recovering after a manual kill (for test purpose) or any runtime crash, bind exception occurs approximately in 50% of the tests/attempts.

This is in spite of closing connection and stopping client inside death listeners/handler (in case of consuming Android service bound with AIDL going down).

Is network service subscription still required? Please let me know how it is done using Jetty websocket package. Is it wrong to use specific IP (static assigned) and port like I showed in previous comment?

port and IP address are specific requirements since the server is another machine that is developed and maintained by 3rd party.

[kprabha7]

I was able to avoid this error in our implementation by configuring some IPv4 sysfs entries on platform. I set the following values:

echo 10 >/proc/sys/net/ipv4/tcp_fin_timeout echo 1 >/proc/sys/net/ipv4/tcp_tw_recycle echo 1 >/proc/sys/net/ipv4/tcp_tw_reuse

After running this, we could see that bind exception didn't occur even after repeated process kills.

Any comments?

[sbordet]

You're probably hitting lack of SO_REUSEADDR which IIRC is controlled by /proc/sys/net/ipv4/tcp_tw_reuse.

I have filed https://github.com/eclipse/jetty.project/issues/6372 to review the options, especially on client-side.

[kprabha7]

Yes @sbordet. I tried to get access to underlying socket object. But I couldn't find any such API. I believe I will be able to set this config for our application without changing global system net config.

If this is added as an API, it will be very useful.

[sbordet]

@kprabha7 in Jetty 9.4.x override the AbstractConnectorHttpClientTransport.configure(HttpClient client, SocketChannel channel) method.

There is a similar method in HTTP2Client if you use HTTP/2.

For Jetty 10, a similar method in ClientConnector.

After #6372, you will be able to call a setter rather than overriding a method.

[janbartel]

Closing this issue in favour of #6372