Is that possible to execute gatling simulations and fail in some case?

[palerique]

Is there a way to execute gatling simulations and fail if some address is unreachable? Or if the app performance is not acceptable?

Thanks, PH

[eshepelyuk]

What happens now to gatlingRun task if destination host is unreacheable ? Does it pass successfully ?

[palerique]

For example:

` ---- Errors --------------------------------------------------------------------

jsonPath($.status).find(0).is(DOWN), but actually found UP 1 (100.0%)

BUILD SUCCESSFUL `

In that case the assertion fails but the build is successful!

Is it possible to fail in that case?

[eshepelyuk]

Ok, makes sense. Let me investigate if there's a way to implement this. Will try to feedback until NY.

[eshepelyuk]

If there're any runtime errors, Gatling produces output similar to this.

 ---- Errors --------------------------------------------------------------------
 > j.n.UnknownHostException: xxx-database.gatling.io                   3 (100,0%)
================================================================================

We can analyze Gatling output and fail gatlingRun (and entire build) if such section occurs.

@palerique @lkishalmi plz feedback

[lkishalmi]

We might add some regexp filter on the output and if that matches a number of lines then let the build fail.

[rakeshnambiar]

My assertion failed. But simulation NOT

.exec(session => { assert(requestStatus.eq("pass")) session })

Is there any way to make the simulation fail? It's kind of a blocker for me

[eshepelyuk]

@rakeshnambiar can you attach logs plz ?

[rakeshnambiar]

@eshepelyuk .. Pls find the log below and sorry for the delay.

INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building literature_performance_tests 2.2.5 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- gatling-maven-plugin:2.2.4:execute (default-cli) @ literature_performance_tests --- 11:35:34,100 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback-test.xml] at [file:/C:/new-git-repository/literature_performance_tests/target/test-classes/logback-test.xml] 11:35:34,146 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set 11:35:34,148 |-INFO in ch.qos.logback.classic.joran.action.LoggerContextListenerAction - Adding LoggerContextListener of type [ch.qos.logback.classic.jul.LevelChangePropagator] to the object stack 11:35:34,160 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@1f256fa - Propagating DEBUG level on Logger[ROOT] onto the JUL framework 11:35:34,160 |-INFO in ch.qos.logback.classic.joran.action.LoggerContextListenerAction - Starting LoggerContextListener 11:35:34,160 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender] 11:35:34,162 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [CONSOLE] 11:35:34,163 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property 11:35:34,166 |-WARN in ch.qos.logback.classic.encoder.PatternLayoutEncoder@cd8737 - As of version 1.2.0 "immediateFlush" property should be set within the enclosing Appender. 11:35:34,166 |-WARN in ch.qos.logback.classic.encoder.PatternLayoutEncoder@cd8737 - Please move "immediateFlush" property into the enclosing appender. 11:35:34,200 |-WARN in ch.qos.logback.classic.encoder.PatternLayoutEncoder@cd8737 - Setting the "immediateFlush" property of the enclosing appender to false 11:35:34,201 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [io.gatling.http] to DEBUG 11:35:34,201 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@1f256fa - Propagating DEBUG level on Logger[io.gatling.http] onto the JUL framework 11:35:34,201 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to DEBUG 11:35:34,201 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [CONSOLE] to Logger[ROOT] 11:35:34,201 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration. 11:35:34,202 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@f71333 - Registering current configuration as safe fallback point

11:35:34.966 [INFO ] a.e.s.Slf4jLogger - Slf4jLogger started 11:35:35.648 [INFO ] i.g.c.s.w.ConsoleDataWriter - Initializing 11:35:35.648 [INFO ] i.g.c.s.w.LogFileDataWriter - Initializing 11:35:35.654 [INFO ] i.g.c.s.w.ConsoleDataWriter - Initialized 11:35:35.693 [INFO ] i.g.c.s.w.LogFileDataWriter - Initialized 11:35:35.721 [DEBUG] i.n.c.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 16 11:35:35.739 [DEBUG] i.n.u.i.PlatformDependent - Platform: Windows 11:35:35.740 [DEBUG] i.n.u.i.PlatformDependent0 - -Dio.netty.noUnsafe: false 11:35:35.740 [DEBUG] i.n.u.i.PlatformDependent0 - Java version: 8 11:35:35.741 [DEBUG] i.n.u.i.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available 11:35:35.741 [DEBUG] i.n.u.i.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available 11:35:35.742 [DEBUG] i.n.u.i.PlatformDependent0 - java.nio.Buffer.address: available 11:35:35.742 [DEBUG] i.n.u.i.PlatformDependent0 - direct buffer constructor: available 11:35:35.742 [DEBUG] i.n.u.i.PlatformDependent0 - java.nio.Bits.unaligned: available, true 11:35:35.742 [DEBUG] i.n.u.i.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9 11:35:35.742 [DEBUG] i.n.u.i.PlatformDependent0 - java.nio.DirectByteBuffer.(long, int): available 11:35:35.742 [DEBUG] i.n.u.i.PlatformDependent - sun.misc.Unsafe: available 11:35:35.743 [DEBUG] i.n.u.i.PlatformDependent - -Dio.netty.tmpdir: C:\Users\RAKESH~1\AppData\Local\Temp (java.io.tmpdir) 11:35:35.743 [DEBUG] i.n.u.i.PlatformDependent - -Dio.netty.bitMode: 32 (sun.arch.data.model) 11:35:35.743 [DEBUG] i.n.u.i.PlatformDependent - -Dio.netty.noPreferDirect: false 11:35:35.744 [DEBUG] i.n.u.i.PlatformDependent - -Dio.netty.maxDirectMemory: 259522560 bytes 11:35:35.744 [DEBUG] i.n.u.i.PlatformDependent - -Dio.netty.uninitializedArrayAllocationThreshold: -1 11:35:35.744 [DEBUG] i.n.u.i.CleanerJava6 - java.nio.ByteBuffer.cleaner(): available 11:35:35.757 [DEBUG] i.n.c.n.NioEventLoop - -Dio.netty.noKeySetOptimization: false 11:35:35.757 [DEBUG] i.n.c.n.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512 11:35:35.769 [DEBUG] i.n.u.i.PlatformDependent - org.jctools-core.MpscChunkedArrayQueue: available 11:35:35.790 [DEBUG] i.n.u.ResourceLeakDetector - -Dio.netty.leakDetection.level: simple 11:35:35.790 [DEBUG] i.n.u.ResourceLeakDetector - -Dio.netty.leakDetection.maxRecords: 4 11:35:35.790 [DEBUG] i.n.u.ResourceLeakDetectorFactory - Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@12aa1a3 11:35:35.818 [DEBUG] i.n.u.i.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024 11:35:35.818 [DEBUG] i.n.u.i.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096 11:35:35.832 [DEBUG] i.n.h.s.JdkSslContext - Default protocols (JDK): [TLSv1.2, TLSv1.1, TLSv1] 11:35:35.832 [DEBUG] i.n.h.s.JdkSslContext - Default cipher suites (JDK): [TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA] 11:35:35.852 [DEBUG] i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.numHeapArenas: 2 11:35:35.852 [DEBUG] i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.numDirectArenas: 2 11:35:35.852 [DEBUG] i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.pageSize: 8192 11:35:35.852 [DEBUG] i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.maxOrder: 11 11:35:35.852 [DEBUG] i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.chunkSize: 16777216 11:35:35.852 [DEBUG] i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.tinyCacheSize: 512 11:35:35.852 [DEBUG] i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.smallCacheSize: 256 11:35:35.852 [DEBUG] i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.normalCacheSize: 64 11:35:35.853 [DEBUG] i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.maxCachedBufferCapacity: 32768 11:35:35.853 [DEBUG] i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.cacheTrimInterval: 8192 11:35:35.853 [DEBUG] i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.useCacheForAllThreads: true 11:35:35.859 [DEBUG] i.n.b.ByteBufUtil - -Dio.netty.allocator.type: pooled 11:35:35.859 [DEBUG] i.n.b.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536 11:35:35.859 [DEBUG] i.n.b.ByteBufUtil - -Dio.netty.maxThreadLocalCharBufferSize: 200000 11:35:35.883 [INFO ] i.g.h.a.HttpEngine - Start warm up 11:35:35.952 [DEBUG] i.n.b.AbstractByteBuf - -Dio.netty.buffer.bytebuf.checkAccessible: true 11:35:35.952 [DEBUG] i.n.u.ResourceLeakDetectorFactory - Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@aba70a 11:35:35.973 [DEBUG] o.a.n.c.NettyConnectListener - Using new Channel '[id: 0x7b367129, L:/172.22.69.59:50774 - R:gatling.io/35.157.221.8:80]' for 'GET' to '/' 11:35:35.979 [DEBUG] i.n.u.Recycler - -Dio.netty.recycler.maxCapacity.default: 32768 11:35:35.979 [DEBUG] i.n.u.Recycler - -Dio.netty.recycler.maxSharedCapacityFactor: 2 11:35:35.979 [DEBUG] i.n.u.Recycler - -Dio.netty.recycler.linkCapacity: 16 11:35:35.979 [DEBUG] i.n.u.Recycler - -Dio.netty.recycler.ratio: 8 11:35:36.004 [DEBUG] o.a.n.h.HttpHandler -

Request DefaultFullHttpRequest(decodeResult: success, version: HTTP/1.1, content: EmptyByteBufBE) GET / HTTP/1.1 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip Connection: keep-alive User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0 Host: gatling.io

Response DefaultHttpResponse(decodeResult: success, version: HTTP/1.1) HTTP/1.1 301 Moved Permanently Server: nginx/1.12.1 Date: Fri, 13 Oct 2017 10:35:36 GMT Content-Type: text/html Content-Length: 185 Connection: keep-alive Location: https://gatling.io/

11:35:36.007 [DEBUG] o.a.n.c.ChannelManager - Adding key: http://gatling.io:80 for channel // Edited 11:35:36.009 [INFO ] i.g.h.a.HttpEngine - Warm up done Simulation SoapSearchPublicationsSnipOFF started... 11:35:36.067 [DEBUG] i.g.c.c.Controller - Setting up max duration 11:35:36.076 [DEBUG] i.g.c.c.i.Injector - Start user #1 11:35:36.080 [DEBUG] i.g.c.c.i.Injector - Injecting 1 users, continue=false 11:35:36.081 [INFO ] i.g.c.c.Controller - InjectionStopped expectedCount=1 // Edited Removed key 11:35:36.441 [DEBUG] o.a.n.h.HttpHandler -

Request DefaultFullHttpRequest(decodeResult: success, version: HTTP/1.1, content: CompositeByteBuf(freed, components=3)) POST /europepmc/webservices/soap HTTP/1.1 SOAPAction: Operation: searchPublications Content-Type: text/xml;charset=UTF-8 Style: Document Host: myhost // Edited Removed key Type: Request-Response Content-Length: 472 Accept: /

Response DefaultHttpResponse(decodeResult: success, version: HTTP/1.1) HTTP/1.1 200 OK Server: Apache-Coyote/1.1 Content-Type: text/xml;charset=utf-8 Strict-Transport-Security: max-age=0 Date: Fri, 13 Oct 2017 10:35:35 GMT Transfer-Encoding: chunked Connection: Keep-Alive Set-Cookie: JSESSIONID=C44B1E84F38716981F4461039DC93E9F; Path=/europepmc/webservices/; HttpOnly

// Edited Removed key 11:35:36.735 [WARN ] i.g.h.a.ResponseProcessor - Encountered error while extracting extra request info java.lang.AssertionError: assertion failed at scala.Predef$.assert(Predef.scala:204) at SoapSearchPublicationsSnipOFF.validateResponse(SoapSearchPublicationsSnipOFF.scala:67) at SoapSearchPublicationsSnipOFF.$anonfun$scn$2(SoapSearchPublicationsSnipOFF.scala:40) at io.gatling.http.ahc.ResponseProcessor.logRequest(ResponseProcessor.scala:113) at io.gatling.http.ahc.ResponseProcessor.logAndExecuteNext(ResponseProcessor.scala:175) at io.gatling.http.ahc.ResponseProcessor.checkAndProceed$1(ResponseProcessor.scala:300) at io.gatling.http.ahc.ResponseProcessor.processResponse(ResponseProcessor.scala:330) at io.gatling.http.ahc.ResponseProcessor.onCompleted(ResponseProcessor.scala:65) at io.gatling.http.ahc.AsyncHandler.$anonfun$onCompleted$1(AsyncHandler.scala:136) at io.gatling.http.ahc.AsyncHandler.$anonfun$onCompleted$1$adapted(AsyncHandler.scala:134) at io.gatling.http.ahc.AsyncHandler.withResponse(AsyncHandler.scala:127) at io.gatling.http.ahc.AsyncHandler.onCompleted(AsyncHandler.scala:134) at io.gatling.http.ahc.AsyncHandler.onCompleted(AsyncHandler.scala:47) at org.asynchttpclient.netty.NettyResponseFuture.getContent(NettyResponseFuture.java:193) at org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:227) at org.asynchttpclient.netty.handler.HttpHandler.finishUpdate(HttpHandler.java:59) at org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:161) at org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:189) at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:76) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1296) at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1087) at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1122) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:491) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:430) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1302) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:581) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) at java.lang.Thread.run(Unknown Source) 11:35:36.737 [DEBUG] i.g.c.a.Exit - End user #1 11:35:36.740 [DEBUG] i.g.c.c.Controller - End user #1 11:35:36.740 [INFO ] i.g.c.c.Controller - All users are stopped 11:35:36.740 [DEBUG] o.a.n.h.HttpHandler - Channel Closed: [id: 0xb3e1ba3b, L:/Ip ! R:host/ip-address] with attribute INSTANCE

================================================================================ 2017-10-13 11:35:36 1s elapsed ---- Requests ------------------------------------------------------------------

Global (OK=1 KO=0 ) Search_Publications (OK=1 KO=0 )

---- SOAP API Without Snippets ------------------------------------------------- [##########################################################################]100% waiting: 0 / active: 0 / done:1

11:35:36.750 [INFO ] i.g.c.c.Controller - StatsEngineStopped Simulation SoapSearchPublicationsSnipOFF completed in 0 seconds 11:35:36.775 [DEBUG] i.n.b.PoolThreadCache - Freed 3 thread-local buffer(s) from thread: gatling-http-thread-1-1 11:35:36.775 [DEBUG] i.n.b.PoolThreadCache - Freed 100 thread-local buffer(s) from thread: gatling-http-thread-1-2 Parsing log file(s)... 11:35:36.796 [INFO ] i.g.c.s.LogFileReader - Collected ArrayBuffer(C:\new-git-repository\literature_performance_tests\target\gatling\results\soapsearchpublicationssnipoff-1507890935633\simulation.log) from soapsearchpublicationssnipoff-1507890935633 11:35:36.807 [INFO ] i.g.c.s.LogFileReader - First pass 11:35:36.824 [INFO ] i.g.c.s.LogFileReader - First pass done: read 5 lines 11:35:36.832 [INFO ] i.g.c.s.LogFileReader - Second pass 11:35:36.850 [INFO ] i.g.c.s.LogFileReader - Second pass: read 5 lines Parsing log file(s) done Generating reports...

================================================================================ ---- Global Information --------------------------------------------------------

request count 1 (OK=1 KO=0 ) min response time 572 (OK=572 KO=- ) max response time 572 (OK=572 KO=- ) mean response time 572 (OK=572 KO=- ) std deviation 0 (OK=0 KO=- ) response time 50th percentile 572 (OK=572 KO=- ) response time 75th percentile 572 (OK=572 KO=- ) response time 95th percentile 572 (OK=572 KO=- ) response time 99th percentile 572 (OK=572 KO=- ) mean requests/sec 1 (OK=1 KO=- ) ---- Response Time Distribution ------------------------------------------------ t < 800 ms 1 (100%) 800 ms < t < 1200 ms 0 ( 0%) t > 1200 ms 0 ( 0%) failed 0 ( 0%)

Reports generated in 0s. Please open the following file: C:\new-git-repository\literature_performance_tests\target\gatling\results\soapsearchpublicationssnipoff-1507890935633\index.html Global: percentage of successful requests is greater than or equal to 100.0 : true [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 8.617 s [INFO] Finished at: 2017-10-13T11:35:37+01:00 [INFO] Final Memory: 6M/17M [INFO] ------------------------------------------------------------------------

[eshepelyuk]

Well, as far as I see all simulation passed successfully, so plugin is unable to detect anything in this case.

[eshepelyuk]

No updates so far, closing the issue.