ConcurrentModificationException while running load test using

[chhil]

20-06-22 10:34:00,024 INFO o.a.j.r.Summariser: Generate Summary Results +   2530 in 00:00:30 =   84.6/s Avg:     0 Min:     0 Max:    51 Err:     0 (0.00%) Active: 100 Started: 100 Finished: 0
2020-06-22 10:34:00,036 INFO o.a.j.r.Summariser: Generate Summary Results =  13691 in 00:02:20 =   97.8/s Avg:     1 Min:     0 Max:   177 Err:     0 (0.00%)
2020-06-22 10:34:30,116 INFO o.a.j.r.Summariser: Generate Summary Results +   2980 in 00:00:30 =   99.0/s Avg:     1 Min:     0 Max:    43 Err:     0 (0.00%) Active: 100 Started: 100 Finished: 0
2020-06-22 10:34:30,125 INFO o.a.j.r.Summariser: Generate Summary Results =  16671 in 00:02:50 =   98.0/s Avg:     1 Min:     0 Max:   177 Err:     0 (0.00%)
2020-06-22 10:35:00,072 INFO o.a.j.r.Summariser: Generate Summary Results +   3080 in 00:00:30 =  102.8/s Avg:     0 Min:     0 Max:    21 Err:     0 (0.00%) Active: 100 Started: 100 Finished: 0
2020-06-22 10:35:00,086 INFO o.a.j.r.Summariser: Generate Summary Results =  19751 in 00:03:20 =   98.7/s Avg:     1 Min:     0 Max:   177 Err:     0 (0.00%)
2020-06-22 10:35:11,538 ERROR o.a.j.t.JMeterThread: Error while processing sampler: '0100 Banknet  Request'.
java.util.ConcurrentModificationException: null
    at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1211) ~[?:1.8.0_152]
    at java.util.TreeMap$KeyIterator.next(TreeMap.java:1265) ~[?:1.8.0_152]
    at org.jpos.iso.ISOMsg.dump(ISOMsg.java:522) ~[jpos-2.1.4.jar:2.1.4]
    at nz.co.breakpoint.jmeter.iso8583.MessagePrinter.asString(MessagePrinter.java:19) ~[jmeter-iso8583-1.1.jar:?]
    at nz.co.breakpoint.jmeter.iso8583.ISO8583Sampler.sample(ISO8583Sampler.java:153) ~[jmeter-iso8583-1.1.jar:?]
    at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:627) ~[ApacheJMeter_core.jar:5.2.1]
    at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:551) ~[ApacheJMeter_core.jar:5.2.1]
    at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:490) [ApacheJMeter_core.jar:5.2.1]
    at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:257) [ApacheJMeter_core.jar:5.2.1]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]

Jmeter jmx zip is attached. If there is any additional info I can provide , please let me know. Visa_MC_Test Plan.zip

[tilln]

Hi @chhil Thanks for reporting this. I have noticed this issue before but not managed to get to the bottom of it yet. Also, concurrency is hard :)

It appears that something is modifying the fields of the message while it is being dumped. At first I thought that MessagePrinter is not thread-safe but there is no shared state.

The jPOS Mux and Channel do modify messages but they don't change fields, just direction, header, and packager, so I'm at a loss where this would actually happen.

I'll see when I can find some time to dig into this. If this is impacting your testing and you need an urgent fix, I'm happy to just either turn off the printing via a config switch or just swallow the exception as it only impacts the message rendering for the JMeter output/report but not the load generation.

[chhil]

It's not impacting my testing and its only happened once. I will reach out to jpos and see if its something that can be resolved there. Feel free to keep this open or close this issue.

[FSchumacher]

It might be a side effect, that the MessageBuilder shares state and you are using it as thread safe, which it is not (in my opinion). I think you might get a reference to a ISOMsg, that is being built for another thread and that might get modified, while you are printing your message.

Hm, but than again, you don't share the builder to other instances of the sampler. That should be safe.

[tilln]

Thanks, @FSchumacher. You are right: MessageBuilder is not thread-safe but should not have to be as there is nothing shared.

I have managed to narrow it down a little and then found a work-around (see #51) that seems to work.