search

vert-x3 / vertx-service-discovery

Some tools one can use for doing microservices with Vert.x
Apache License 2.0
115 stars 67 forks source link

Intermittent failure in KubernetesServerTest.testWatch #95

Closed vietj closed 6 years ago

vietj commented 6 years ago 
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running io.vertx.servicediscovery.kubernetes.OpenShiftServerTest
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
MockWebServer[53519] starting to accept connections 
Kubernetes discovery configured for namespace: default 
Kubernetes master url: http//localhost:53519 
Kubernetes discovery: Bearer Token { some-token } 
MockWebServer[53519] received request: GET /api/v1/namespaces/default/services HTTP/1.1 and responded: HTTP/1.1 200 OK 
Kubernetes initial import of 2 services 
Kubernetes service published in the vert.x service registry: {"location":{"port":1524,"targetPort":8080,"internal-port":8080,"host":null},"metadata":{"kubernetes.namespace":"my-project","kubernetes.name":"my-service","kubernetes.uuid":"uuid-2"},"name":"my-service","registration":"86836306-2053-4496-a294-e66cb06b3260","status":"UP","type":"unknown"} 
Kubernetes service published in the vert.x service registry: {"location":{"endpoint":"http://null:8080","port":8080,"root":"","ssl":false},"metadata":{"service-type":"http-endpoint","kubernetes.namespace":"my-project","kubernetes.name":"my-http-service","kubernetes.uuid":"uuid-1"},"name":"my-http-service","registration":"efffbfae-2488-4356-8f69-015ca8de2b05","status":"UP","type":"http-endpoint"} 
Kubernetes importer instantiated with 2 services imported 
Service importer io.vertx.servicediscovery.kubernetes.KubernetesServiceImporter@d308524 started 
MockWebServer[53519] received request: GET /api/v1/namespaces/default/services?watch=true&resourceVersion=1234 HTTP/1.1 and responded: HTTP/1.1 200 OK 
Adding service service3 
Kubernetes service published in the vert.x service registry: {"location":{"port":1524,"targetPort":8080,"internal-port":8080,"host":null},"metadata":{"kubernetes.namespace":"my-project","kubernetes.name":"service3","kubernetes.uuid":"uid-3"},"name":"service3","registration":"28713f13-12c0-47e3-ae1d-8b54222f2c8c","status":"UP","type":"unknown"} 
Removing service my-service 
Kubernetes service unregistered from the vert.x registry: {"location":{"port":1524,"targetPort":8080,"internal-port":8080,"host":null},"metadata":{"kubernetes.namespace":"my-project","kubernetes.name":"my-service","kubernetes.uuid":"uuid-2"},"name":"my-service","registration":"86836306-2053-4496-a294-e66cb06b3260","status":"UP","type":"unknown"} 
Modifying service my-http-service 
Kubernetes service published in the vert.x service registry: {"location":{"endpoint":"http://null:8080","port":8080,"root":"","ssl":false},"metadata":{"service-type":"http-endpoint","foo":"bar","kubernetes.namespace":"my-project","kubernetes.name":"my-http-service","kubernetes.uuid":"uuid-1"},"name":"my-http-service","registration":"636bd236-54ea-4f2e-9129-7af281211304","status":"UP","type":"http-endpoint"} 
Watching services from namespace default 
MockWebServer[56470] starting to accept connections 
Kubernetes discovery configured for namespace: default 
Kubernetes master url: http//localhost:56470 
Kubernetes discovery: Bearer Token { some-token } 
MockWebServer[56470] received request: GET /api/v1/namespaces/default/services HTTP/1.1 and responded: HTTP/1.1 200 OK 
Kubernetes initial import of 2 services 
Kubernetes service published in the vert.x service registry: {"location":{"port":1524,"targetPort":8080,"internal-port":8080,"host":null},"metadata":{"kubernetes.namespace":"my-project","kubernetes.name":"my-service","kubernetes.uuid":"uuid-2"},"name":"my-service","registration":"5574a8fb-1f1f-49d5-9eb0-f53149c318cf","status":"UP","type":"unknown"} 
Kubernetes service published in the vert.x service registry: {"location":{"endpoint":"http://null:8080","port":8080,"root":"","ssl":false},"metadata":{"service-type":"http-endpoint","kubernetes.namespace":"my-project","kubernetes.name":"my-http-service","kubernetes.uuid":"uuid-1"},"name":"my-http-service","registration":"2a154b8c-7e64-4a61-93d8-25351e7d6b25","status":"UP","type":"http-endpoint"} 
Kubernetes importer instantiated with 2 services imported 
Service importer io.vertx.servicediscovery.kubernetes.KubernetesServiceImporter@296495a7 started 
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.784 sec - in io.vertx.servicediscovery.kubernetes.OpenShiftServerTest
Running io.vertx.servicediscovery.kubernetes.KubernetesServiceImporterTest
MockWebServer[56470] received request: GET /api/v1/namespaces/default/services?watch=true&resourceVersion=1234 HTTP/1.1 and responded: HTTP/1.1 200 OK 
Adding service service3 
Kubernetes service published in the vert.x service registry: {"location":{"port":1524,"targetPort":8080,"internal-port":8080,"host":null},"metadata":{"kubernetes.namespace":"my-project","kubernetes.name":"service3","kubernetes.uuid":"uid-3"},"name":"service3","registration":"a35352e1-fa43-4a00-855c-b9187814c791","status":"UP","type":"unknown"} 
Removing service my-service 
Kubernetes service unregistered from the vert.x registry: {"location":{"port":1524,"targetPort":8080,"internal-port":8080,"host":null},"metadata":{"kubernetes.namespace":"my-project","kubernetes.name":"my-service","kubernetes.uuid":"uuid-2"},"name":"my-service","registration":"5574a8fb-1f1f-49d5-9eb0-f53149c318cf","status":"UP","type":"unknown"} 
Modifying service my-http-service 
Kubernetes service published in the vert.x service registry: {"location":{"endpoint":"http://null:8080","port":8080,"root":"","ssl":false},"metadata":{"service-type":"http-endpoint","foo":"bar","kubernetes.namespace":"my-project","kubernetes.name":"my-http-service","kubernetes.uuid":"uuid-1"},"name":"my-http-service","registration":"cbd2e26d-a415-41e1-be04-41bfc0057355","status":"UP","type":"http-endpoint"} 
Watching services from namespace default 
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.08 sec - in io.vertx.servicediscovery.kubernetes.KubernetesServiceImporterTest
Running io.vertx.servicediscovery.kubernetes.KubernetesServerTest
MockWebServer[36425] starting to accept connections 
Kubernetes discovery configured for namespace: default 
Kubernetes master url: http//localhost:36425 
Kubernetes discovery: Bearer Token { some-token } 
MockWebServer[36425] received request: GET /api/v1/namespaces/default/services HTTP/1.1 and responded: HTTP/1.1 200 OK 
Kubernetes initial import of 2 services 
Kubernetes service published in the vert.x service registry: {"location":{"port":1524,"targetPort":8080,"internal-port":8080,"host":null},"metadata":{"kubernetes.namespace":"my-project","kubernetes.name":"my-service","kubernetes.uuid":"uuid-2"},"name":"my-service","registration":"4e966882-2fc4-47bf-9704-d044f33f330e","status":"UP","type":"unknown"} 
Kubernetes service published in the vert.x service registry: {"location":{"endpoint":"http://null:8080","port":8080,"root":"","ssl":false},"metadata":{"service-type":"http-endpoint","kubernetes.namespace":"my-project","kubernetes.name":"my-http-service","kubernetes.uuid":"uuid-1"},"name":"my-http-service","registration":"fa15710d-d3ef-441d-956c-e68e31d24d68","status":"UP","type":"http-endpoint"} 
Kubernetes importer instantiated with 2 services imported 
Service importer io.vertx.servicediscovery.kubernetes.KubernetesServiceImporter@55587e9a started 
MockWebServer[36425] received request: GET /api/v1/namespaces/default/services?watch=true&resourceVersion=1234 HTTP/1.1 and responded: HTTP/1.1 200 OK 
Adding service service3 
Kubernetes service published in the vert.x service registry: {"location":{"port":1524,"targetPort":8080,"internal-port":8080,"host":null},"metadata":{"kubernetes.namespace":"my-project","kubernetes.name":"service3","kubernetes.uuid":"uid-3"},"name":"service3","registration":"c342c38b-d18b-4c8c-8c56-00737f35e850","status":"UP","type":"unknown"} 
Unable to setup the watcher on the service list 
io.vertx.core.json.DecodeException: Failed to decode: Unexpected end-of-input in field name
 at [Source: (String)"{"object":{"apiVersion":"v1","kind":"Service","metadata":{"finalizers":[],"name":"my-service","namespace":"my-project","ownerReferences":[],"uid":"uuid-2"},"spec":{"deprecatedPublicIPs":[],"externalIPs":[],"loadBal"; line: 1, column: 429]
    at io.vertx.core.json.Json.decodeValue(Json.java:120)
    at io.vertx.core.json.JsonObject.fromJson(JsonObject.java:956)
    at io.vertx.core.json.JsonObject.<init>(JsonObject.java:48)
    at io.vertx.servicediscovery.kubernetes.KubernetesServiceImporter$1.lambda$write$0(KubernetesServiceImporter.java:188)
    at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
    at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
    at io.vertx.servicediscovery.kubernetes.KubernetesServiceImporter$1.write(KubernetesServiceImporter.java:188)
    at io.vertx.servicediscovery.kubernetes.KubernetesServiceImporter$1.write(KubernetesServiceImporter.java:179)
    at io.vertx.ext.web.codec.impl.StreamingBodyCodec$1.write(StreamingBodyCodec.java:63)
    at io.vertx.ext.web.codec.impl.StreamingBodyCodec$1.write(StreamingBodyCodec.java:39)
    at io.vertx.core.streams.impl.PumpImpl.lambda$new$1(PumpImpl.java:59)
    at io.vertx.core.http.impl.HttpClientResponseImpl.handleChunk(HttpClientResponseImpl.java:211)
    at io.vertx.core.http.impl.Http1xClientConnection$StreamImpl.lambda$beginResponse$0(Http1xClientConnection.java:454)
    at io.vertx.core.queue.impl.BufferedQueue.safeHandle(BufferedQueue.java:88)
    at io.vertx.core.queue.impl.BufferedQueue.add(BufferedQueue.java:68)
    at io.vertx.core.http.impl.Http1xClientConnection$StreamImpl.handleChunk(Http1xClientConnection.java:310)
    at io.vertx.core.http.impl.Http1xClientConnection$StreamImpl.access$1600(Http1xClientConnection.java:191)
    at io.vertx.core.http.impl.Http1xClientConnection.handleResponseChunk(Http1xClientConnection.java:616)
    at io.vertx.core.http.impl.Http1xClientConnection.handleHttpMessage(Http1xClientConnection.java:579)
    at io.vertx.core.http.impl.Http1xClientConnection.handleMessage(Http1xClientConnection.java:540)
    at io.vertx.core.net.impl.ConnectionBase.handleRead(ConnectionBase.java:353)
    at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:333)
    at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:188)
    at io.vertx.core.impl.EventLoopContext.executeFromIO(EventLoopContext.java:23)
    at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:157)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:646)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:581)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:460)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)

Kubernetes discovery configured for namespace: default 
Kubernetes master url: http//localhost:36628 
Kubernetes discovery: Bearer Token { some-token } 
MockWebServer[36628] starting to accept connections 
MockWebServer[36628] received request: GET /api/v1/namespaces/default/services HTTP/1.1 and responded: HTTP/1.1 200 OK 
Kubernetes initial import of 2 services 
Kubernetes service published in the vert.x service registry: {"location":{"port":1524,"targetPort":8080,"internal-port":8080,"host":null},"metadata":{"kubernetes.namespace":"my-project","kubernetes.name":"my-service","kubernetes.uuid":"uuid-2"},"name":"my-service","registration":"25aff634-756d-4adc-a080-285860c2ebee","status":"UP","type":"unknown"} 
Kubernetes service published in the vert.x service registry: {"location":{"endpoint":"http://null:8080","port":8080,"root":"","ssl":false},"metadata":{"service-type":"http-endpoint","kubernetes.namespace":"my-project","kubernetes.name":"my-http-service","kubernetes.uuid":"uuid-1"},"name":"my-http-service","registration":"40b65e8f-ecf2-482a-bd58-99989cb3ca73","status":"UP","type":"http-endpoint"} 
Kubernetes importer instantiated with 2 services imported 
Service importer io.vertx.servicediscovery.kubernetes.KubernetesServiceImporter@68e079ef started 
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 10.066 sec <<< FAILURE! - in io.vertx.servicediscovery.kubernetes.KubernetesServerTest
testWatch(io.vertx.servicediscovery.kubernetes.KubernetesServerTest)  Time elapsed: 10.038 sec  <<< ERROR!
com.jayway.awaitility.core.ConditionTimeoutException: Condition defined as a lambda expression in io.vertx.servicediscovery.kubernetes.KubernetesServerTest Found service 'my-service' in the list within 10 seconds.
    at com.jayway.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:122)
    at com.jayway.awaitility.core.AssertionCondition.await(AssertionCondition.java:117)
    at com.jayway.awaitility.core.AssertionCondition.await(AssertionCondition.java:32)
    at com.jayway.awaitility.core.ConditionFactory.until(ConditionFactory.java:764)
    at com.jayway.awaitility.core.ConditionFactory.until(ConditionFactory.java:603)
    at io.vertx.servicediscovery.kubernetes.KubernetesServerTest.testWatch(KubernetesServerTest.java:107)

Results :

Tests in error: 
  KubernetesServerTest.testWatch:107 » ConditionTimeout Condition defined as a l...

Tests run: 8, Failures: 0, Errors: 1, Skipped: 0

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project vertx-service-discovery-bridge-kubernetes: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/travis/build/vert-x3/vertx-service-discovery/vertx-service-discovery-bridge-kubernetes/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :vertx-service-discovery-bridge-kubernetes
vietj commented 6 years ago

it seems still to happen in Travis : https://api.travis-ci.org/v3/job/447996407/log.txt

cescoffier commented 6 years ago

Fixed... This time for good!

vietj commented 6 years ago

🎉🎉🎉🎉🎉🎉🎉

vietj commented 6 years ago

one step more step to CI Nirvana