Closed arinban closed 7 years ago
@LucianoGandini Commented Thanks for taking care of the merge @rlubke, can we get a 2.3.x release soon? We need to test it and validate it for next release.
@rlubke Commented @LucianoGandini https://javaee.groups.io/g/grizzly/message/22
On org.glassfish.grizzly.nio.NIOConnection, CloseListener objects are stored in a ConcurrentLinkedQueue, but it seems that it is not used as a Queue and objects are removed unorderly (without taking in consideration the especial usage in notifyCloseListeners) using the remove(Object) method.
What does the remove method do?
As you can see it linearly seeks for the object to be removed from the queue, but the Node is not deleted, causing future deletions to iterate over and over old Nodes, this Nodes appear to be removed when a FullGC is run but in the meantime causes the deletion of the entry to cost more and more. In our test scenarios it takes ~30 minutes to run the FullGC and at least 18M Nodes are still there as you can see in the following image:![screen shot 2017-08-04 at 5 34 58 pm](https://user-images.githubusercontent.com/2371470/28986071-e4934e3c-793b-11e7-8816-4409d925f97b.png)
This behaviour impacts our applications behaviour reducing performance over time with very high impact, here is our tps variation on a 1 hour test:
And its correlation to GC's:
![pasted image at 2017_07_13 12_07 pm](https://user-images.githubusercontent.com/2371470/28986273-9c554ea8-793c-11e7-9072-277a3b1ba3a6.png)
I've replace this collection with a Set as its not needed to use the queue for any particular reason and the tps does not degrade anymore but it stays linear over time.
This issue was found using Grizzly 2.3.29, I'll send a PR for 2.4.x also as code will be cleaner in java8.