Many threads creation by SyncIdleConnectionMonitorThread

[KopeMorta]

I have to work with a large number of Unirest instances and I had a problem of creating new threads. The problem is the kong.unirest.apache.SyncIdleConnectionMonitorThread class. For each instance of Uniest, a new thread of this class is created. And in my case, jvm crashes after creating ~ 16kk threads.

To fix this problem, I think need to use thread pools. For example, Executors.newCachedThreadPool ().

Of course I close all instances. At the same time, usually no more than 1k instances work.

I understand that problem is quite specific :)

=>0x00000000819cf000 JavaThread "Thread-16135980" daemon [_thread_in_vm, id=636100, stack(0x000000007f0d0000,0x000000007f1d0000)]

Register to memory mapping:

RAX=0x000000007f1cc2c0 is pointing into the stack for thread: 0x00000000819cf000
RBX=0x0000000000000004 is an unknown value
RCX=0x0000000000003000 is an unknown value
RDX=0x0000000000000006 is an unknown value
RSP=0x000000007f1cf2c0 is pointing into the stack for thread: 0x00000000819cf000
RBP=0x0000000000000000 is an unknown value
RSI=0x000000007f1cf2c0 is pointing into the stack for thread: 0x00000000819cf000
RDI=0x0000000000000004 is an unknown value
R8 =0x0000000000000006 is an unknown value
R9 =0x000000003b702f50 is an unknown value
R10={method} {0x000000004d7eccf8} 'run' '()V' in 'kong/unirest/apache/SyncIdleConnectionMonitorThread'
R11=0x000000007f0da170 is pointing into the stack for thread: 0x00000000819cf000
R12=0x000000003b702f50 is an unknown value
R13=0x00000000819cf000 is a thread
R14=0x000000007f1cf388 is pointing into the stack for thread: 0x00000000819cf000
R15={method} {0x000000004d7eccf8} 'run' '()V' in 'kong/unirest/apache/SyncIdleConnectionMonitorThread'

Stack: [0x000000007f0d0000,0x000000007f1d0000],  sp=0x000000007f1cf2c0,  free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

[ryber]

Why do you have so many configured instances of Unirest? It's designed such that you should have relatively few in a particular JVM. What is stopping you from re-using them?

[KopeMorta]

Why do you have so many configured instances of Unirest? It's designed such that you should have relatively few in a particular JVM. What is stopping you from re-using them?

At a minimum, I need to clear cookies and I don’t know how to do this. But also I have no idea what each instance stores in itself, maybe it accumulates some information.

[ryber]

You can disable cookie tracking with Unirest.config().enableCookieManagement(false); the rest of what's "tracked" is just security stuff. I couple possible enable not even having the thread, but that might have other weird side effects.

[KopeMorta]

You can disable cookie tracking with Unirest.config().enableCookieManagement(false);

I need cookie management. But I need to clear the cookies. Now I am doing this by creating new instances. And there is no other way, as it seemed to me.

the rest of what's "tracked" is just security stuff.

We saw that there was a thread at number 16kk. About 30 requests are executed for one thread. And if at least a little data is collect there, it seems to me that this is just a memory leak (in my case).

I couple possible enable not even having the thread, but that might have other weird side effects.

I did not understand this sentence. Do you mean that you have a way to create instances without a monitor ? If so, this is most likely my option.

[ryber]

You can certainly populate Unirest with your own configured instance of Apache Http Client that doesn't use a threadmonitor and has whatever you need as far as cookies go:

https://github.com/Kong/unirest-java/blob/master/unirest/src/test/java/BehaviorTests/CustomClientTest.java#L57-L81

[KopeMorta]

You can certainly populate Unirest with your own configured instance of Apache Http Client that doesn't use a threadmonitor and has whatever you need as far as cookies go:

https://github.com/Kong/unirest-java/blob/master/unirest/src/test/java/BehaviorTests/CustomClientTest.java#L57-L81

At first I thought: “What can I change if the client is created immediately, when building the config”. And then I checked - a very interesting solution, building a client on demand.

Well then you can consider the problem settled. Thanks for the help.