search

ballerina-platform / ballerina-lang

The Ballerina Programming Language
https://ballerina.io/
Apache License 2.0
3.59k stars 740 forks source link

Servers becomes unresponsive when the transaction scenario is executed using Vegeta tool #8525

Closed sashikamw closed 3 years ago

sashikamw commented 6 years ago

Description: It was observed that the servers getting unresponsive when the transaction scenario is executed using Vegeta tool. Only a limited amount of requests got successful.

Steps to reproduce:

  1. Use the artefacts in [1] and set up the ballerina services [1] https://github.com/afkham/microtransactions/blob/master/src/initiator/initiator.bal
  2. Execute the service using vegeta tool echo "GET http://192.168.11.11:8888" | ./vegeta attack -duration=100s | tee results.bin | ./vegeta report

Affected Versions: ballerina-0.970.0-rc1 and ballerina-runtime-0.970.0

OS, DB, other environment details and versions: Ubuntu 15.10

Observations Following errors were observed in the following ballerina services initiator.bal

[2018-04-27 17:01:31,865] WARN  {org.wso2.transport.http.netty.sender.TargetHandler} - Idle timeout has reached hence closing the connection 2a818176  
[2018-04-27 17:01:31,870] WARN  {org.wso2.transport.http.netty.sender.TargetHandler} - Idle timeout has reached hence closing the connection 9ce244de

Participant1.bal 

[2018-04-27 17:01:30,724] WARN  {org.wso2.transport.http.netty.sender.TargetHandler} - Idle timeout has reached hence closing the connection 6814375b  
[2018-04-27 17:01:30,724] ERROR {org.ballerinalang.net.http.HttpUtil} - call failed  
[2018-04-27 17:01:30,727] ERROR {org.wso2.transport.http.netty.common.Util} - Remote client closed the connection before completing outbound response 
java.io.IOException: Remote client closed the connection before completing outbound response
    at org.wso2.transport.http.netty.common.Util.lambda$checkForResponseWriteStatus$9(Util.java:599)
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
    at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
    at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:163)
    at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93)
    at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:28)
    at org.wso2.transport.http.netty.common.Util.checkForResponseWriteStatus(Util.java:595)
    at org.wso2.transport.http.netty.contractimpl.HttpOutboundRespListener.writeOutboundResponseHeaderAndBody(HttpOutboundRespListener.java:187)
    at org.wso2.transport.http.netty.contractimpl.HttpOutboundRespListener.writeOutboundResponse(HttpOutboundRespListener.java:138)
    at org.wso2.transport.http.netty.contractimpl.HttpOutboundRespListener.lambda$null$35(HttpOutboundRespListener.java:94)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:745)

Participnat2.bal

[2018-04-27 17:01:30,685] ERROR {org.wso2.transport.http.netty.common.Util} - Remote client closed the connection before completing outbound response 
java.io.IOException: Remote client closed the connection before completing outbound response
    at org.wso2.transport.http.netty.common.Util.lambda$checkForResponseWriteStatus$9(Util.java:599)
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
    at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
    at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:163)
    at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93)
    at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:28)
    at org.wso2.transport.http.netty.common.Util.checkForResponseWriteStatus(Util.java:595)
    at org.wso2.transport.http.netty.contractimpl.HttpOutboundRespListener.writeOutboundResponseHeaderAndBody(HttpOutboundRespListener.java:187)
    at org.wso2.transport.http.netty.contractimpl.HttpOutboundRespListener.writeOutboundResponse(HttpOutboundRespListener.java:138)
    at org.wso2.transport.http.netty.contractimpl.HttpOutboundRespListener.lambda$null$35(HttpOutboundRespListener.java:94)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:745)

Vegeta Tool outcome

wso2@ubuntu:~/VEGETA$ echo "GET http://10.100.8.9:8080" | ./vegeta attack -duration=120s | tee results.bin | ./vegeta report
Requests      [total, rate]            6000, 50.01
Duration      [total, attack, wait]    2m22.261331689s, 1m59.979999615s, 22.281332074s
Latencies     [mean, 50, 95, 99, max]  20.411990535s, 30.001887206s, 30.007298356s, 30.150733445s, 31.080372027s
Bytes In      [total, mean]            0, 0.00
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  0.02%
Status Codes  [code:count]             200:1  0:5999  
Error Set:
Get http://10.100.8.9:8080: dial tcp 0.0.0.0:0->10.100.8.9:8080: socket: too many open files
Get http://10.100.8.9:8080: net/http: timeout awaiting response headers

Note - This issue did not observe with the jmeter

warunalakshitha commented 3 years ago

We have done significant design changes to transaction implementation and syntax in swan lake hence above scenario is outdated. I am closing this issue please reopen it if you have any concerns on this.