search

kyma-project / kyma

Kyma is an opinionated set of Kubernetes-based modular building blocks, including all necessary capabilities to develop and run enterprise-grade cloud-native applications.
https://kyma-project.io
Apache License 2.0
1.52k stars 405 forks source link

Metadata tests are unstable #1572

Closed akgalwas closed 6 years ago

akgalwas commented 6 years ago

Metadata tests fail once in a while on cluster.

Investigate the root cause and file follow up issue is needed.

mjasinski5 commented 6 years ago

Logs from NIghtly:

tests:

2018/11/07 17:15:15 Read configuration: {MetadataServiceUrl:http://metadata-service-external-api.kyma-integration.svc.cluster.local:8081 Namespace:kyma-integration}
--- FAIL: TestK8sResources (15.81s)
    --- FAIL: TestK8sResources/when_updating_service_and_removing_API (12.14s)
        metadata_api_client.go:208:
            --------------------------------
            HTTP/1.1 200 OK
            Content-Length: 46
            Content-Type: application/json;charset=UTF-8
            Date: Wed, 07 Nov 2018 17:15:18 GMT
            Server: envoy
            X-Envoy-Decorator-Operation: metadata-service-external-api.kyma-integration.svc.cluster.local:8081/*
            X-Envoy-Upstream-Service-Time: 331
            {"id":"d54c5f5c-1c6b-4209-a660-8419948d4c65"}
            --------------------------------
        metadata_api_client.go:208:
            --------------------------------
            HTTP/1.1 500 Internal Server Error
            Content-Length: 39
            Content-Type: application/json;charset=UTF-8
            Date: Wed, 07 Nov 2018 17:15:27 GMT
            Server: envoy
            X-Envoy-Decorator-Operation: metadata-service-external-api.kyma-integration.svc.cluster.local:8081/*
            X-Envoy-Upstream-Service-Time: 91
            {"code":500,"error":"Internal error."}
            --------------------------------
        require.go:157:
                Error Trace:    k8sresource_test.go:338
                Error:          Not equal:
                                expected: 200
                                actual  : 500
                Test:           TestK8sResources/when_updating_service_and_removing_API
FAIL

metadatservice (when test was faliling):

time="2018-11-07T17:15:09Z" level=error msg="Updating service failed, Updating 12345 service failed, Service with ID 12345 not found" remote environment=dummy-re service ID=12345
time="2018-11-07T17:15:10Z" level=error msg="Getting service by ID failed, Service with ID 3cf38edc-31cc-49ad-800a-3e327daef5e8 not found" remote environment=dummy-re service ID=3cf38edc-31cc-49ad-800a-3e327daef5e8
time="2018-11-07T17:15:18Z" level=error msg="Updating service failed, Updating d54c5f5c-1c6b-4209-a660-8419948d4c65 service failed, updating service in Remote Environment repository failed, Updating service failed, Operation cannot be fulfilled on remoteenvironments.applicationconnector.kyma-project.io \"dummy-re\": the object has been modified; please apply your changes to the latest version and try again" remote environment=dummy-re service ID=d54c5f5c-1c6b-4209-a660-8419948d4c65
time="2018-11-07T17:15:23Z" level=error msg="Updating service failed, Updating d54c5f5c-1c6b-4209-a660-8419948d4c65 service failed, Reading API failed, Reading oauth credentials from re-dummy-re-d54c5f5c-1c6b-4209-a660-8419948d4c65 secret failed, Secret re-dummy-re-d54c5f5c-1c6b-4209-a660-8419948d4c65 not found" remote environment=dummy-re service ID=d54c5f5c-1c6b-4209-a660-8419948d4c65
time="2018-11-07T17:15:28Z" level=error msg="Updating service failed, Updating d54c5f5c-1c6b-4209-a660-8419948d4c65 service failed, Reading API failed, Reading oauth credentials from re-dummy-re-d54c5f5c-1c6b-4209-a660-8419948d4c65 secret failed, Secret re-dummy-re-d54c5f5c-1c6b-4209-a660-8419948d4c65 not found" remote environment=dummy-re service ID=d54c5f5c-1c6b-4209-a660-8419948d4c65

all logs from that period from metadata service:

time="2018-11-07T17:15:01Z" level=info msg="GET /ec-default/v1/metadata/services HTTP/1.1\r\nHost: gateway.nightly.cluster.kyma.cx\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nSsl-Client-Issuer-Dn: CN=wormhole.kyma.cx,OU=Kyma,O=SAP Hybris,L=GLIWICE,ST=N,C=PL\r\nSsl-Client-Subject-Dn: CN=ec-default,OU=OrgUnit,O=Organization,L=Waldorf,ST=Waldorf,C=DE\r\nSsl-Client-Verify: SUCCESS\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Flags: 0\r\nX-B3-Parentspanid: 70b74b102591efe9\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 9e4c1b31759da68a\r\nX-B3-Traceid: f08efcdf0c2fd56e\r\nX-Envoy-Internal: true\r\nX-Forwarded-For: 10.240.0.6\r\nX-Forwarded-Host: gateway.nightly.cluster.kyma.cx\r\nX-Forwarded-Port: 443\r\nX-Forwarded-Proto: https\r\nX-Original-Uri: /ec-default/v1/metadata/services\r\nX-Real-Ip: 10.240.0.6\r\nX-Request-Id: cbe3aed63820f6633cc5d38579a9042a\r\nX-Scheme: https\r\n\r\n" remote environment=ec-default
time="2018-11-07T17:15:01Z" level=info msg="Services read successfully." remote environment=ec-default
time="2018-11-07T17:15:06Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 547\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: cbcf19ed2cd718f5\r\nX-B3-Traceid: cbcf19ed2cd718f5\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 8a379271-2afd-9480-8d19-3081394d4c31\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:07Z" level=info msg="Service with ID 269970b0-4841-4f6b-a4c2-ecf36aa5ceff created successfully." remote environment=dummy-re
time="2018-11-07T17:15:07Z" level=info msg="GET /dummy-re/v1/metadata/services/269970b0-4841-4f6b-a4c2-ecf36aa5ceff HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: fd656829a05bfc93\r\nX-B3-Traceid: fd656829a05bfc93\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 2fe94589-3f33-92a5-9c37-5fb450c4f897\r\n\r\n" remote environment=dummy-re service ID=269970b0-4841-4f6b-a4c2-ecf36aa5ceff
time="2018-11-07T17:15:07Z" level=info msg="Service read successfully." remote environment=dummy-re service ID=269970b0-4841-4f6b-a4c2-ecf36aa5ceff
time="2018-11-07T17:15:07Z" level=info msg="GET /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 6a701e359328ff9e\r\nX-B3-Traceid: 6a701e359328ff9e\r\nX-Forwarded-Proto: http\r\nX-Request-Id: fb34ab92-0b58-9416-ad57-3bad281eedee\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:07Z" level=info msg="Services read successfully." remote environment=dummy-re
time="2018-11-07T17:15:07Z" level=info msg="DELETE /dummy-re/v1/metadata/services/269970b0-4841-4f6b-a4c2-ecf36aa5ceff HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 9d42b37d13dfb43e\r\nX-B3-Traceid: 9d42b37d13dfb43e\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 199f8360-2166-9769-ab84-5ca5cf04e907\r\n\r\n" remote environment=dummy-re service ID=269970b0-4841-4f6b-a4c2-ecf36aa5ceff
time="2018-11-07T17:15:07Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=269970b0-4841-4f6b-a4c2-ecf36aa5ceff
time="2018-11-07T17:15:07Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 591\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 1854b48f726d8233\r\nX-B3-Traceid: 1854b48f726d8233\r\nX-Forwarded-Proto: http\r\nX-Request-Id: ea08b7b6-8730-9bb1-9544-b9dcde600469\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:07Z" level=info msg="Service with ID e8c05f46-5f33-4115-a659-4894a87a476d created successfully." remote environment=dummy-re
time="2018-11-07T17:15:07Z" level=info msg="GET /dummy-re/v1/metadata/services/e8c05f46-5f33-4115-a659-4894a87a476d HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: b89403e07b1fa1ec\r\nX-B3-Traceid: b89403e07b1fa1ec\r\nX-Forwarded-Proto: http\r\nX-Request-Id: cf3f661c-1b30-95bc-b055-237bd1706a56\r\n\r\n" remote environment=dummy-re service ID=e8c05f46-5f33-4115-a659-4894a87a476d
time="2018-11-07T17:15:07Z" level=info msg="Service read successfully." remote environment=dummy-re service ID=e8c05f46-5f33-4115-a659-4894a87a476d
time="2018-11-07T17:15:07Z" level=info msg="GET /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: d5ef9e0b63ed1c60\r\nX-B3-Traceid: d5ef9e0b63ed1c60\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 9a70c0bf-3f10-95ef-b796-eee5039b7ba9\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:07Z" level=info msg="Services read successfully." remote environment=dummy-re
time="2018-11-07T17:15:07Z" level=info msg="DELETE /dummy-re/v1/metadata/services/e8c05f46-5f33-4115-a659-4894a87a476d HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 4382cab732d4e722\r\nX-B3-Traceid: 4382cab732d4e722\r\nX-Forwarded-Proto: http\r\nX-Request-Id: fd737cf1-967e-9cc6-b137-5f69043e338d\r\n\r\n" remote environment=dummy-re service ID=e8c05f46-5f33-4115-a659-4894a87a476d
time="2018-11-07T17:15:07Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=e8c05f46-5f33-4115-a659-4894a87a476d
time="2018-11-07T17:15:07Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 549\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 227434e31abceade\r\nX-B3-Traceid: 227434e31abceade\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 38e9acb2-1572-9a92-8a1b-62f6d6274dba\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:08Z" level=info msg="Service with ID f0fe8772-ddd1-4652-a6b1-7a398dfd9be6 created successfully." remote environment=dummy-re
time="2018-11-07T17:15:08Z" level=info msg="PUT /dummy-re/v1/metadata/services/f0fe8772-ddd1-4652-a6b1-7a398dfd9be6 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 535\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 385c5f66ec711150\r\nX-B3-Traceid: 385c5f66ec711150\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 3e3d8509-5029-972c-998f-87709665e545\r\n\r\n" remote environment=dummy-re service ID=f0fe8772-ddd1-4652-a6b1-7a398dfd9be6
time="2018-11-07T17:15:08Z" level=info msg="Service updated successfully." remote environment=dummy-re service ID=f0fe8772-ddd1-4652-a6b1-7a398dfd9be6
time="2018-11-07T17:15:08Z" level=info msg="GET /dummy-re/v1/metadata/services/f0fe8772-ddd1-4652-a6b1-7a398dfd9be6 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 691a5af483e13dfc\r\nX-B3-Traceid: 691a5af483e13dfc\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 9072e2a4-0153-946c-a411-cd29eba49f6c\r\n\r\n" remote environment=dummy-re service ID=f0fe8772-ddd1-4652-a6b1-7a398dfd9be6
time="2018-11-07T17:15:09Z" level=info msg="Service read successfully." remote environment=dummy-re service ID=f0fe8772-ddd1-4652-a6b1-7a398dfd9be6
time="2018-11-07T17:15:09Z" level=info msg="GET /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 9f03df19d885da13\r\nX-B3-Traceid: 9f03df19d885da13\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 06679609-71f6-9617-acc1-5decdafaf46c\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:09Z" level=info msg="Services read successfully." remote environment=dummy-re
time="2018-11-07T17:15:09Z" level=info msg="DELETE /dummy-re/v1/metadata/services/f0fe8772-ddd1-4652-a6b1-7a398dfd9be6 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 24a02044fd9bde2c\r\nX-B3-Traceid: 24a02044fd9bde2c\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 7f9e97a3-5ddc-9e29-8999-28bd124b5105\r\n\r\n" remote environment=dummy-re service ID=f0fe8772-ddd1-4652-a6b1-7a398dfd9be6
time="2018-11-07T17:15:09Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=f0fe8772-ddd1-4652-a6b1-7a398dfd9be6
time="2018-11-07T17:15:09Z" level=info msg="PUT /dummy-re/v1/metadata/services/12345 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 535\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: a23272b51eef65b9\r\nX-B3-Traceid: a23272b51eef65b9\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 04f77027-b31d-9044-a455-4804a3c68a3e\r\n\r\n" remote environment=dummy-re service ID=12345
time="2018-11-07T17:15:09Z" level=error msg="Updating service failed, Updating 12345 service failed, Service with ID 12345 not found" remote environment=dummy-re service ID=12345
time="2018-11-07T17:15:09Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 549\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: d605bdc85bcbf7ca\r\nX-B3-Traceid: d605bdc85bcbf7ca\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 0fa260f8-74dd-9f05-be02-1ccf82fbc937\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:10Z" level=info msg="Service with ID 3cf38edc-31cc-49ad-800a-3e327daef5e8 created successfully." remote environment=dummy-re
time="2018-11-07T17:15:10Z" level=info msg="DELETE /dummy-re/v1/metadata/services/3cf38edc-31cc-49ad-800a-3e327daef5e8 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: de8136d237a23452\r\nX-B3-Traceid: de8136d237a23452\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 75c01335-f3ce-9d2f-811e-37d4dc268766\r\n\r\n" remote environment=dummy-re service ID=3cf38edc-31cc-49ad-800a-3e327daef5e8
time="2018-11-07T17:15:10Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=3cf38edc-31cc-49ad-800a-3e327daef5e8
time="2018-11-07T17:15:10Z" level=info msg="GET /dummy-re/v1/metadata/services/3cf38edc-31cc-49ad-800a-3e327daef5e8 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 37eae59e9d262025\r\nX-B3-Traceid: 37eae59e9d262025\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 83d0c12b-2c58-9dfc-afe1-47eb6da60696\r\n\r\n" remote environment=dummy-re service ID=3cf38edc-31cc-49ad-800a-3e327daef5e8
time="2018-11-07T17:15:10Z" level=error msg="Getting service by ID failed, Service with ID 3cf38edc-31cc-49ad-800a-3e327daef5e8 not found" remote environment=dummy-re service ID=3cf38edc-31cc-49ad-800a-3e327daef5e8
time="2018-11-07T17:15:11Z" level=info msg="GET /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: a94259dd5245c6f4\r\nX-B3-Traceid: a94259dd5245c6f4\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 0275f8d9-bb35-939b-8c7b-c39ecaf8a52f\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:11Z" level=info msg="Services read successfully." remote environment=dummy-re
time="2018-11-07T17:15:11Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 549\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: d1ae6b0c011583e3\r\nX-B3-Traceid: d1ae6b0c011583e3\r\nX-Forwarded-Proto: http\r\nX-Request-Id: bb949ce1-ddcf-9e21-aa7f-b5a9c36115f5\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:11Z" level=info msg="Service with ID 8e434994-d31d-4791-b6d1-1c9d60b79a56 created successfully." remote environment=dummy-re
time="2018-11-07T17:15:11Z" level=info msg="GET /dummy-re/v1/metadata/services/8e434994-d31d-4791-b6d1-1c9d60b79a56 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: d0e7cf3076d04bdc\r\nX-B3-Traceid: d0e7cf3076d04bdc\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 19f8ad82-d4fa-9d6e-a0cc-16d78d480004\r\n\r\n" remote environment=dummy-re service ID=8e434994-d31d-4791-b6d1-1c9d60b79a56
time="2018-11-07T17:15:12Z" level=info msg="Service read successfully." remote environment=dummy-re service ID=8e434994-d31d-4791-b6d1-1c9d60b79a56
time="2018-11-07T17:15:12Z" level=info msg="GET /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 3220031aef76c0e7\r\nX-B3-Traceid: 3220031aef76c0e7\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 27f7a1b8-2954-9cc3-98d7-0469e7626701\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:12Z" level=info msg="Services read successfully." remote environment=dummy-re
time="2018-11-07T17:15:12Z" level=info msg="DELETE /dummy-re/v1/metadata/services/8e434994-d31d-4791-b6d1-1c9d60b79a56 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 882c2f16e3d6ccc8\r\nX-B3-Traceid: 882c2f16e3d6ccc8\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 6f94131e-6b6e-9b07-a06c-f06aae0f9424\r\n\r\n" remote environment=dummy-re service ID=8e434994-d31d-4791-b6d1-1c9d60b79a56
time="2018-11-07T17:15:12Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=8e434994-d31d-4791-b6d1-1c9d60b79a56
time="2018-11-07T17:15:12Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 549\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 9a213542e2601fe2\r\nX-B3-Traceid: 9a213542e2601fe2\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 7d978400-e6c4-9c67-a1ee-98fdea0619c5\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:13Z" level=info msg="Service with ID f0fb5250-6561-4458-96fe-5a13d917cc37 created successfully." remote environment=dummy-re
time="2018-11-07T17:15:13Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 549\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 539a492d8d042fa8\r\nX-B3-Traceid: 539a492d8d042fa8\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 5a1efea8-d3df-97f8-9ee5-23e1e96acb8d\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:13Z" level=info msg="Service with ID 6ed2aaf9-2af9-43a7-b17d-fa8e6841b065 created successfully." remote environment=dummy-re
time="2018-11-07T17:15:13Z" level=info msg="GET /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 3374f8c23e64bacf\r\nX-B3-Traceid: 3374f8c23e64bacf\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 40847ae0-9d79-983c-b691-838dc918fcb2\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:13Z" level=info msg="Services read successfully." remote environment=dummy-re
time="2018-11-07T17:15:14Z" level=info msg="DELETE /dummy-re/v1/metadata/services/f0fb5250-6561-4458-96fe-5a13d917cc37 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: ff0034f3b53985bd\r\nX-B3-Traceid: ff0034f3b53985bd\r\nX-Forwarded-Proto: http\r\nX-Request-Id: f6683844-56c3-91a3-8480-a54910d81022\r\n\r\n" remote environment=dummy-re service ID=f0fb5250-6561-4458-96fe-5a13d917cc37
time="2018-11-07T17:15:14Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=f0fb5250-6561-4458-96fe-5a13d917cc37
time="2018-11-07T17:15:14Z" level=info msg="DELETE /dummy-re/v1/metadata/services/6ed2aaf9-2af9-43a7-b17d-fa8e6841b065 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 44748795252e0b18\r\nX-B3-Traceid: 44748795252e0b18\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 1fb8c543-92d1-9231-b0b1-0dda6bad8151\r\n\r\n" remote environment=dummy-re service ID=6ed2aaf9-2af9-43a7-b17d-fa8e6841b065
time="2018-11-07T17:15:14Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=6ed2aaf9-2af9-43a7-b17d-fa8e6841b065
time="2018-11-07T17:15:15Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 255\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 175182bcae3b7f3c\r\nX-B3-Traceid: 175182bcae3b7f3c\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 2e83c38d-30c6-9618-aa62-9df0e329e09e\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:15Z" level=info msg="Service with ID 25265ba1-15fa-4485-94e7-ce95d8f7fc98 created successfully." remote environment=dummy-re
time="2018-11-07T17:15:15Z" level=info msg="DELETE /dummy-re/v1/metadata/services/25265ba1-15fa-4485-94e7-ce95d8f7fc98 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 837c92929e187666\r\nX-B3-Traceid: 837c92929e187666\r\nX-Forwarded-Proto: http\r\nX-Request-Id: c1ede527-8070-9d79-a027-fb54a043197a\r\n\r\n" remote environment=dummy-re service ID=25265ba1-15fa-4485-94e7-ce95d8f7fc98
time="2018-11-07T17:15:15Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=25265ba1-15fa-4485-94e7-ce95d8f7fc98
time="2018-11-07T17:15:15Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 125\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 73bd535084039441\r\nX-B3-Traceid: 73bd535084039441\r\nX-Forwarded-Proto: http\r\nX-Request-Id: cfef0a54-652a-9d33-ae4c-cb840e771b00\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:16Z" level=info msg="Service with ID 1ca95d5a-bd17-471d-9375-f0ab5ae899ed created successfully." remote environment=dummy-re
time="2018-11-07T17:15:16Z" level=info msg="DELETE /dummy-re/v1/metadata/services/1ca95d5a-bd17-471d-9375-f0ab5ae899ed HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: aa345f0b0190b888\r\nX-B3-Traceid: aa345f0b0190b888\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 9706dcee-9519-98a8-a2ca-58cd630038c0\r\n\r\n" remote environment=dummy-re service ID=1ca95d5a-bd17-471d-9375-f0ab5ae899ed
time="2018-11-07T17:15:16Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=1ca95d5a-bd17-471d-9375-f0ab5ae899ed
time="2018-11-07T17:15:16Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 255\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: fb5744f8b216ea5f\r\nX-B3-Traceid: fb5744f8b216ea5f\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 000d3a2b-1b53-907a-89ab-a55ec495b1c8\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:16Z" level=info msg="Service with ID a90cadf7-1611-49ce-8284-d18e250c8a99 created successfully." remote environment=dummy-re
time="2018-11-07T17:15:16Z" level=info msg="PUT /dummy-re/v1/metadata/services/a90cadf7-1611-49ce-8284-d18e250c8a99 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 347\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: e4a37093fab2da5d\r\nX-B3-Traceid: e4a37093fab2da5d\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 157eccc9-253f-9786-9025-498e5b668dcb\r\n\r\n" remote environment=dummy-re service ID=a90cadf7-1611-49ce-8284-d18e250c8a99
time="2018-11-07T17:15:17Z" level=info msg="Service updated successfully." remote environment=dummy-re service ID=a90cadf7-1611-49ce-8284-d18e250c8a99
time="2018-11-07T17:15:17Z" level=info msg="DELETE /dummy-re/v1/metadata/services/a90cadf7-1611-49ce-8284-d18e250c8a99 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 4ae726d77683d7c8\r\nX-B3-Traceid: 4ae726d77683d7c8\r\nX-Forwarded-Proto: http\r\nX-Request-Id: e433aa9c-7bd8-9be5-a2e7-bcd7b6caa5e4\r\n\r\n" remote environment=dummy-re service ID=a90cadf7-1611-49ce-8284-d18e250c8a99
time="2018-11-07T17:15:17Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=a90cadf7-1611-49ce-8284-d18e250c8a99
time="2018-11-07T17:15:17Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 255\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 5f9dba89ad1c8e89\r\nX-B3-Traceid: 5f9dba89ad1c8e89\r\nX-Forwarded-Proto: http\r\nX-Request-Id: dc09e62d-bad7-96dc-b0e4-79a5054bd4c5\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:18Z" level=info msg="Service with ID d54c5f5c-1c6b-4209-a660-8419948d4c65 created successfully." remote environment=dummy-re
time="2018-11-07T17:15:18Z" level=info msg="PUT /dummy-re/v1/metadata/services/d54c5f5c-1c6b-4209-a660-8419948d4c65 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 149\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 4245d58a5588f389\r\nX-B3-Traceid: 4245d58a5588f389\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 6bd7941e-56a1-96ce-bd35-b8b317bf9a36\r\n\r\n" remote environment=dummy-re service ID=d54c5f5c-1c6b-4209-a660-8419948d4c65
time="2018-11-07T17:15:18Z" level=error msg="Updating service failed, Updating d54c5f5c-1c6b-4209-a660-8419948d4c65 service failed, updating service in Remote Environment repository failed, Updating service failed, Operation cannot be fulfilled on remoteenvironments.applicationconnector.kyma-project.io \"dummy-re\": the object has been modified; please apply your changes to the latest version and try again" remote environment=dummy-re service ID=d54c5f5c-1c6b-4209-a660-8419948d4c65
time="2018-11-07T17:15:23Z" level=info msg="PUT /dummy-re/v1/metadata/services/d54c5f5c-1c6b-4209-a660-8419948d4c65 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 149\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 49aef7625b89370a\r\nX-B3-Traceid: 49aef7625b89370a\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 49f4134f-ab6d-9914-9624-9547ecf29031\r\n\r\n" remote environment=dummy-re service ID=d54c5f5c-1c6b-4209-a660-8419948d4c65
time="2018-11-07T17:15:23Z" level=error msg="Updating service failed, Updating d54c5f5c-1c6b-4209-a660-8419948d4c65 service failed, Reading API failed, Reading oauth credentials from re-dummy-re-d54c5f5c-1c6b-4209-a660-8419948d4c65 secret failed, Secret re-dummy-re-d54c5f5c-1c6b-4209-a660-8419948d4c65 not found" remote environment=dummy-re service ID=d54c5f5c-1c6b-4209-a660-8419948d4c65
time="2018-11-07T17:15:27Z" level=info msg="PUT /dummy-re/v1/metadata/services/d54c5f5c-1c6b-4209-a660-8419948d4c65 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 149\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: e68db28aedfd24c9\r\nX-B3-Traceid: e68db28aedfd24c9\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 04b4d50f-14fe-9f72-b15e-a7b243c78c71\r\n\r\n" remote environment=dummy-re service ID=d54c5f5c-1c6b-4209-a660-8419948d4c65
time="2018-11-07T17:15:28Z" level=error msg="Updating service failed, Updating d54c5f5c-1c6b-4209-a660-8419948d4c65 service failed, Reading API failed, Reading oauth credentials from re-dummy-re-d54c5f5c-1c6b-4209-a660-8419948d4c65 secret failed, Secret re-dummy-re-d54c5f5c-1c6b-4209-a660-8419948d4c65 not found" remote environment=dummy-re service ID=d54c5f5c-1c6b-4209-a660-8419948d4c65
time="2018-11-07T17:15:30Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 125\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: fe53ce915cc47f2b\r\nX-B3-Traceid: fe53ce915cc47f2b\r\nX-Forwarded-Proto: http\r\nX-Request-Id: d745f043-4ac4-9bbf-9fb5-54d6514e0878\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:30Z" level=info msg="Service with ID 3d72cfe7-5606-473c-9b7b-749736c1143d created successfully." remote environment=dummy-re
time="2018-11-07T17:15:30Z" level=info msg="PUT /dummy-re/v1/metadata/services/3d72cfe7-5606-473c-9b7b-749736c1143d HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 279\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 4976138963d2a7c4\r\nX-B3-Traceid: 4976138963d2a7c4\r\nX-Forwarded-Proto: http\r\nX-Request-Id: d59604bc-b4ce-9e1e-8278-01a9ca4af9a1\r\n\r\n" remote environment=dummy-re service ID=3d72cfe7-5606-473c-9b7b-749736c1143d
time="2018-11-07T17:15:30Z" level=info msg="Service updated successfully." remote environment=dummy-re service ID=3d72cfe7-5606-473c-9b7b-749736c1143d
time="2018-11-07T17:15:30Z" level=info msg="DELETE /dummy-re/v1/metadata/services/3d72cfe7-5606-473c-9b7b-749736c1143d HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: 38d84e61d3887240\r\nX-B3-Traceid: 38d84e61d3887240\r\nX-Forwarded-Proto: http\r\nX-Request-Id: 53ec2a3d-a905-954e-bf47-f874bc22b443\r\n\r\n" remote environment=dummy-re service ID=3d72cfe7-5606-473c-9b7b-749736c1143d
time="2018-11-07T17:15:30Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=3d72cfe7-5606-473c-9b7b-749736c1143d
time="2018-11-07T17:15:30Z" level=info msg="POST /dummy-re/v1/metadata/services HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 291\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: a60130a3f6f9c3a6\r\nX-B3-Traceid: a60130a3f6f9c3a6\r\nX-Forwarded-Proto: http\r\nX-Request-Id: a1576a15-e72b-9ffe-8caf-c01db1717e65\r\n\r\n" remote environment=dummy-re
time="2018-11-07T17:15:30Z" level=info msg="Service with ID c3041b57-2e1d-4e48-8ec3-deb3d3644e47 created successfully." remote environment=dummy-re
time="2018-11-07T17:15:30Z" level=info msg="DELETE /dummy-re/v1/metadata/services/c3041b57-2e1d-4e48-8ec3-deb3d3644e47 HTTP/1.1\r\nHost: metadata-service-external-api.kyma-integration.svc.cluster.local:8081\r\nAccept-Encoding: gzip\r\nContent-Length: 0\r\nUser-Agent: Go-http-client/1.1\r\nX-B3-Sampled: 1\r\nX-B3-Spanid: bc4d6e214e9f9568\r\nX-B3-Traceid: bc4d6e214e9f9568\r\nX-Forwarded-Proto: http\r\nX-Request-Id: d648c193-7dcb-9a67-beb0-9db31c6a6441\r\n\r\n" remote environment=dummy-re service ID=c3041b57-2e1d-4e48-8ec3-deb3d3644e47
time="2018-11-07T17:15:30Z" level=info msg="Service deleted successfully." remote environment=dummy-re service ID=c3041b57-2e1d-4e48-8ec3-deb3d3644e47