What steps will reproduce the problem?
1.Create a diameter client
2.Start it on a machine with a lot of cores
3.Start will fail occasionally
What is the expected output? What do you see instead?
Diameter stack should start. Instead, follow exception is thrown:
org.jdiameter.api.InternalException: java.lang.IllegalArgumentException:
Statistic already defined: Statistic{ records=[0, 0.0] }
at org.jdiameter.client.impl.controller.PeerImpl.connect(PeerImpl.java:457)
at org.jdiameter.client.impl.controller.PeerTableImpl.start(PeerTableImpl.java:275)
at org.jdiameter.client.impl.StackImpl.startPeerManager(StackImpl.java:246)
at org.jdiameter.client.impl.StackImpl.start(StackImpl.java:221)
at ..............................
Caused by: java.lang.IllegalArgumentException: Statistic already defined:
Statistic{ records=[0, 0.0] }
at org.jdiameter.common.impl.statistic.StatisticManagerImpl.newStatistic(StatisticManagerImpl.java:130)
at org.jdiameter.client.impl.fsm.PeerFSMImpl.runQueueProcessing(PeerFSMImpl.java:177)
at org.jdiameter.client.impl.fsm.PeerFSMImpl.handleEvent(PeerFSMImpl.java:294)
at org.jdiameter.client.impl.controller.PeerImpl.connect(PeerImpl.java:454)
... 42 more
What version of the product are you using? On what operating system?
1.5.10.0, but the same problem exists in the latest source as well
Please provide any additional information below.
Root cause is the check for numberOfThreadsRunning in PeerFSMImpl. There is a
lock to prevent concurrent access to the numberOfThreadsRunning, but the count
is increased ouside of this lock (by another thread). So if an event happens
(eg START event) before the numberOfThreadsRunning has been increased, the
runQueueuProcessing is executed twice leading to the above error.
This can be seen in the logs as well. The 'Starting FSM Thread' messages are
printed first, but at that point numberOfThreadsRunning is not increased yet.
That only happens when 'FSM threads are running' messages are shown (after the
exception occured)
[main] DEBUG org.jdiameter.client.impl.controller.PeerTableImpl - Initializing
Peer Table.
[main] DEBUG org.jdiameter.client.impl.controller.PeerTableImpl - Populating
peerTable from configuration
[main] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl - Maximum FSM Queue size
is [10000]
[main] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl - Initializing QueueStat
@ Thread[main]
[main] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl - Finished Initializing
QueueStat @ Thread[main]
[main] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl - Starting FSM Thread 1
of 3
[main] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl - Starting FSM Thread 2
of 3
[main] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl - Starting FSM Thread 3
of 3
[main] DEBUG org.jdiameter.client.impl.transport.tcp.TCPTransportClient -
Destination address is set to [127.0.0.1] : [3868]
[main] DEBUG org.jdiameter.client.impl.transport.tcp.TCPTransportClient -
Origin address is set to [0.0.0.0] : [0]
[main] DEBUG org.jdiameter.client.impl.controller.PeerTableImpl - Appended peer
[CPeer{Uri=aaa://127.0.0.1:3868; State=DOWN;
Con=org.jdiameter.client.impl.transport.tcp.TCPClientConnection@11e70f62}] to
peer table
[main] INFO org.jdiameter.client.impl.StackImpl - (-)(-)(-)(-)(-) Started
Mobicents DIAMETER Stack v1.5.10.0-build639 (-)(-)(-)(-)(-)
[main] DEBUG org.jdiameter.client.impl.controller.PeerTableImpl - Starting
PeerTable. Going to call connect on all peers in the peerTable
[main] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl - Handling event with
type [START_EVENT]
[main] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl - No FSM threads are
running so calling runQueueProcessing()
[main] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl - Initializing QueueStat
@ Thread[main]
[main] WARN org.jdiameter.client.impl.controller.PeerTableImpl - Can not start
connect procedure to peer [CPeer{Uri=aaa://127.0.0.1:3868; State=DOWN;
Con=org.jdiameter.client.impl.transport.tcp.TCPClientConnection@11e70f62}]
org.jdiameter.api.InternalException: java.lang.IllegalArgumentException:
Statistic already defined: Statistic{ records=[0, 0.0] }
[main] DEBUG org.jdiameter.client.impl.controller.PeerTableImpl - Calling start
on the router
[FSM-aaa://127.0.0.1:3868_2-1] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl
- Starting ... [3] FSM threads are running
[FSM-aaa://127.0.0.1:3868_1-0] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl
- Starting ... [2] FSM threads are running
[FSM-aaa://127.0.0.1:3868_3-2] DEBUG org.jdiameter.client.impl.fsm.PeerFSMImpl
- Starting ... [1] FSM threads are running
Original issue reported on code.google.com by janss...@gmail.com on 27 Jan 2015 at 9:26
Original issue reported on code.google.com by
janss...@gmail.comon 27 Jan 2015 at 9:26