gerritjvv / bigstreams

bigstreams big data kafka hadoop and file based imports
Eclipse Public License 1.0
3 stars 3 forks source link

Collector GC 2 minutes on ParNew (30,430 collections) #13

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago
Collector garbage collection is 2 minutes for ParNew

Possible solution is to set the number of concurrent threads lower.

see http://forums.sun.com/thread.jspa?threadID=5244426

Quote:
After the exciting reply I did get on my question, we did some more 
investigations on the problem and it seems that we finally found the solution 
to our problem.

The number of garbage collection threads used by the virtual machine defaults 
to the number of cpus of the machine.
This is ok for small machines or machines where the main load is produced by 
the java application itself.
In our environment the main load is not produced by the java application but 
oracle database processes.
When java tries to do it's garbage collection using 120 threads (# CPU) on the 
machine which is already overloaded by non java processes, the thread 
synchronization seems to produce an exorbitant overhead.
My theory is that spin locking is used on memory access, causing threads to 
spin while waiting for other blocking threads not getting cpu because of the 
heavy load on the system.
The solution is now to limit the number of garbage collection threads.

We did that on the first try by setting -XX:ParallelGCThreads=8

Original issue reported on code.google.com by gerritjvv@gmail.com on 27 Sep 2010 at 4:14

GoogleCodeExporter commented 8 years ago

Original comment by gerritjvv@gmail.com on 29 Sep 2010 at 3:16