search

openshift / origin

Conformance test suite for OpenShift
http://www.openshift.org
Apache License 2.0
8.49k stars 4.7k forks source link

Extended.[templates] templateinstance impersonation tests should pass impersonation update tests fails #15211

Closed gabemontero closed 7 years ago

gabemontero commented 7 years ago

@openshift/devex FYI

Not obvious which scenario within this test failed (there is an array of several data inputs going through this loop). Some more debug may be warranted.

@jim-minter I've initially assigned to you, but we can discuss either here or in scrum to-do's (updates to the test case, possible inferences from the data available from the data here that I'm missing, etc.) and who does what.

Saw no prior flakes along these lines.

First instance: https://ci.openshift.redhat.com/jenkins/job/test_branch_origin_extended_templates/19/

Stacktrace

/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/templates/templateinstance_impersonation.go:221
Expected an error to have occurred.  Got:
    <nil>: nil
/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/templates/templateinstance_impersonation.go:205

Standard Output

[BeforeEach] [Top Level]
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/test.go:52
[BeforeEach] [templates] templateinstance impersonation tests
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:120
�[1mSTEP�[0m: Creating a kubernetes client
Jul 14 09:34:05.392: INFO: >>> kubeConfig: /etc/origin/master/admin.kubeconfig

�[1mSTEP�[0m: Building a namespace api object
Jul 14 09:34:05.411: INFO: configPath is now "/tmp/extended-test-templates-x0sp6-71zl5-user.kubeconfig"
Jul 14 09:34:05.411: INFO: The user is now "extended-test-templates-x0sp6-71zl5-user"
Jul 14 09:34:05.411: INFO: Creating project "extended-test-templates-x0sp6-71zl5"
�[1mSTEP�[0m: Waiting for a default service account to be provisioned in namespace
[BeforeEach] [templates] templateinstance impersonation tests
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/templates/templateinstance_impersonation.go:151
[It] should pass impersonation update tests
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/templates/templateinstance_impersonation.go:221
�[1mSTEP�[0m: testing as system:admin user
�[1mSTEP�[0m: testing as extended-test-templates-x0sp6-71zl5-adminuser user
Jul 14 09:34:05.797: INFO: configPath is now "/tmp/extended-test-templates-x0sp6-71zl5-adminuser.kubeconfig"
[AfterEach] [templates] templateinstance impersonation tests
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:121
�[1mSTEP�[0m: Collecting events from namespace "extended-test-templates-x0sp6-71zl5".
�[1mSTEP�[0m: Found 0 events.
Jul 14 09:34:05.822: INFO: POD                       NODE                        PHASE    GRACE  CONDITIONS
Jul 14 09:34:05.822: INFO: docker-registry-1-9x0nd   ip-172-18-0-7.ec2.internal  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-07-14 09:19:27 -0400 EDT  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-07-14 09:19:47 -0400 EDT  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-07-14 09:19:27 -0400 EDT  }]
Jul 14 09:34:05.822: INFO: registry-console-1-sjp4j  ip-172-18-0-7.ec2.internal  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-07-14 09:20:08 -0400 EDT  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-07-14 09:21:38 -0400 EDT  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-07-14 09:20:08 -0400 EDT  }]
Jul 14 09:34:05.822: INFO: router-1-29tj1            ip-172-18-0-7.ec2.internal  Running         [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-07-14 09:18:36 -0400 EDT  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-07-14 09:18:56 -0400 EDT  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-07-14 09:18:36 -0400 EDT  }]
Jul 14 09:34:05.822: INFO: 
Jul 14 09:34:05.825: INFO: 
Logging node info for node ip-172-18-0-7.ec2.internal
Jul 14 09:34:05.827: INFO: Node Info: &Node{ObjectMeta:k8s_io_apimachinery_pkg_apis_meta_v1.ObjectMeta{Name:ip-172-18-0-7.ec2.internal,GenerateName:,Namespace:,SelfLink:/api/v1/nodes/ip-172-18-0-7.ec2.internal,UID:9cf2b0c1-6896-11e7-9a74-0ebacd4fe67c,ResourceVersion:1968,Generation:0,CreationTimestamp:2017-07-14 09:16:14 -0400 EDT,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{beta.kubernetes.io/arch: amd64,beta.kubernetes.io/os: linux,kubernetes.io/hostname: ip-172-18-0-7.ec2.internal,region: infra,zone: default,},Annotations:map[string]string{volumes.kubernetes.io/controller-managed-attach-detach: true,},OwnerReferences:[],Finalizers:[],ClusterName:,},Spec:NodeSpec{PodCIDR:,ExternalID:ip-172-18-0-7.ec2.internal,ProviderID:aws:////i-0feb999df8950c34b,Unschedulable:false,Taints:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{4 0} {<nil>} 4 DecimalSI},memory: {{16388657152 0} {<nil>} 16004548Ki BinarySI},pods: {{40 0} {<nil>} 40 DecimalSI},},Allocatable:ResourceList{cpu: {{4 0} {<nil>} 4 DecimalSI},memory: {{16283799552 0} {<nil>} 15902148Ki BinarySI},pods: {{40 0} {<nil>} 40 DecimalSI},},Phase:,Conditions:[{OutOfDisk False 2017-07-14 09:34:00 -0400 EDT 2017-07-14 09:16:14 -0400 EDT KubeletHasSufficientDisk kubelet has sufficient disk space available} {MemoryPressure False 2017-07-14 09:34:00 -0400 EDT 2017-07-14 09:16:14 -0400 EDT KubeletHasSufficientMemory kubelet has sufficient memory available} {DiskPressure False 2017-07-14 09:34:00 -0400 EDT 2017-07-14 09:16:14 -0400 EDT KubeletHasNoDiskPressure kubelet has no disk pressure} {Ready True 2017-07-14 09:34:00 -0400 EDT 2017-07-14 09:16:27 -0400 EDT KubeletReady kubelet is posting ready status}],Addresses:[{LegacyHostIP 172.18.0.7} {InternalIP 172.18.0.7} {Hostname ip-172-18-0-7.ec2.internal}],DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f9370ed252a14f73b014c1301a9b6d1b,SystemUUID:EC2050E5-7059-D722-142E-35ED45238386,BootID:a64d0db5-363c-47d9-9d66-a2cd4f21f801,KernelVersion:3.10.0-514.26.2.el7.x86_64,OSImage:Red Hat Enterprise Linux Server 7.3 (Maipo),ContainerRuntimeVersion:docker://1.12.6,KubeletVersion:v1.6.1+5115d708d7,KubeProxyVersion:v1.6.1+5115d708d7,OperatingSystem:linux,Architecture:amd64,},Images:[{[openshift/origin-federation:6ba4bf5 openshift/origin-federation:latest] 1162326050} {[openshift/origin-docker-registry:6ba4bf5 openshift/origin-docker-registry:latest] 1058352753} {[openshift/origin-gitserver:6ba4bf5 openshift/origin-gitserver:latest] 1044758106} {[openshift/openvswitch:6ba4bf5 openshift/openvswitch:latest] 1023024304} {[openshift/node:6ba4bf5 openshift/node:latest] 1021342609} {[openshift/origin-keepalived-ipfailover:6ba4bf5 openshift/origin-keepalived-ipfailover:latest] 998138366} {[openshift/origin-service-catalog:6ba4bf5 openshift/origin-service-catalog:latest] 996845926} {[openshift/origin-haproxy-router:6ba4bf5 openshift/origin-haproxy-router:latest] 992373497} {[openshift/origin:6ba4bf5 openshift/origin:latest] 971349513} {[openshift/origin-sti-builder:6ba4bf5 openshift/origin-sti-builder:latest] 971349513} {[openshift/origin-docker-builder:6ba4bf5 openshift/origin-docker-builder:latest] 971349513} {[openshift/origin-recycler:6ba4bf5 openshift/origin-recycler:latest] 971349513} {[openshift/origin-deployer:6ba4bf5 openshift/origin-deployer:latest] 971349513} {[openshift/origin-f5-router:6ba4bf5 openshift/origin-f5-router:latest] 971349513} {[openshift/origin-cluster-capacity:6ba4bf5 openshift/origin-cluster-capacity:latest] 932070315} {[docker.io/openshift/origin-release@sha256:a7286ac59e96ef58db24a4563767f1eebe6c1088cd4b2252e2be923d2dcef742 docker.io/openshift/origin-release:golang-1.7] 861277957} {[openshift/origin-logging-auth-proxy:27d11de openshift/origin-logging-auth-proxy:latest] 668551000} {[docker.io/node@sha256:5757581a8ff7e08041512a54aa3f573d33fecdce81d603e48a759956cd99bdd3 docker.io/node:4.7.2] 650142332} {[openshift/origin-logging-kibana:27d11de openshift/origin-logging-kibana:latest] 548422645} {[docker.io/cockpit/kubernetes@sha256:86c54baed25788404dd22441d32a55a78588325159afb26678a764f46f4f6374 docker.io/cockpit/kubernetes:latest] 400184643} {[openshift/origin-egress-http-proxy:6ba4bf5 openshift/origin-egress-http-proxy:latest] 395970189} {[openshift/origin-logging-elasticsearch:27d11de openshift/origin-logging-elasticsearch:latest] 393902110} {[docker.io/openshift/base-centos7@sha256:aea292a3bddba020cde0ee83e6a45807931eb607c164ec6a3674f67039d8cd7c docker.io/openshift/base-centos7:latest] 383049978} {[openshift/origin-egress-router:6ba4bf5 openshift/origin-egress-router:latest] 364717657} {[openshift/origin-base:6ba4bf5 openshift/origin-base:latest] 363042119} {[openshift/origin-logging-fluentd:27d11de openshift/origin-logging-fluentd:latest] 360461829} {[openshift/origin-logging-curator:27d11de openshift/origin-logging-curator:latest] 224967998} {[openshift/origin-pod:6ba4bf5 openshift/origin-pod:latest] 213237298} {[openshift/origin-source:6ba4bf5 openshift/origin-source:latest] 192530021} {[docker.io/centos@sha256:c1010e2fe2b635822d99a096b1f4184becf5d1c98707cbccae00be663a9b9131 docker.io/centos:7 docker.io/centos:centos7] 192529651} {[openshift/hello-openshift:6ba4bf5 openshift/hello-openshift:latest] 5635113}],VolumesInUse:[],VolumesAttached:[],},}
Jul 14 09:34:05.827: INFO: 
Logging kubelet events for node ip-172-18-0-7.ec2.internal
Jul 14 09:34:05.830: INFO: 
Logging pods the kubelet thinks is on node ip-172-18-0-7.ec2.internal
Jul 14 09:34:05.851: INFO: router-1-29tj1 started at 2017-07-14 09:18:36 -0400 EDT (0+1 container statuses recorded)
Jul 14 09:34:05.851: INFO:  Container router ready: true, restart count 0
Jul 14 09:34:05.851: INFO: docker-registry-1-9x0nd started at 2017-07-14 09:19:27 -0400 EDT (0+1 container statuses recorded)
Jul 14 09:34:05.851: INFO:  Container registry ready: true, restart count 0
Jul 14 09:34:05.851: INFO: registry-console-1-sjp4j started at 2017-07-14 09:20:08 -0400 EDT (0+1 container statuses recorded)
Jul 14 09:34:05.851: INFO:  Container registry-console ready: true, restart count 0
Jul 14 09:34:05.946: INFO: 
Latency metrics for node ip-172-18-0-7.ec2.internal
�[1mSTEP�[0m: Dumping a list of prepulled images on each node
Jul 14 09:34:05.949: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
�[1mSTEP�[0m: Destroying namespace "extended-test-templates-x0sp6-71zl5" for this suite.
Jul 14 09:34:16.072: INFO: namespace: extended-test-templates-x0sp6-71zl5, resource: bindings, ignored listing per whitelist
[AfterEach] [templates] templateinstance impersonation tests
  /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/templates/templateinstance_impersonation.go:159
gabemontero commented 7 years ago

This has occurred with https://ci.openshift.redhat.com/jenkins/job/test_branch_origin_extended_templates/20/ and https://ci.openshift.redhat.com/jenkins/job/test_branch_origin_extended_templates/21/ as well.

Perhaps we should change the flake label to a bug one ?

bparees commented 7 years ago

Yeah seems like something has broken this.

Ben Parees | OpenShift

On Jul 16, 2017 12:11 PM, "Gabe Montero" notifications@github.com wrote:

This has occurred with https://ci.openshift.redhat. com/jenkins/job/test_branch_origin_extended_templates/20/ and https://ci.openshift.redhat.com/jenkins/job/test_branch_ origin_extended_templates/21/ as well.

Perhaps we should change the flake label to a bug one ?

— You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub https://github.com/openshift/origin/issues/15211#issuecomment-315619319, or mute the thread https://github.com/notifications/unsubscribe-auth/AEvl3ti7_1MfpP7aCjKmWJ_dLz29ZOCmks5sOjYpgaJpZM4OYd2N .

jim-minter commented 7 years ago

Not obvious which scenario within this test failed (there is an array of several data inputs going through this loop). Some more debug may be warranted.

It's the adminuser test: "STEP: testing as extended-test-templates-x0sp6-71zl5-adminuser user"

jim-minter commented 7 years ago

The code wasn't broken, but the test had become inadequate after (I think) #15112.

gabemontero commented 7 years ago

@jim-minter - thx for the ptr ... my cursory scan of the code did not pick up on the fact that helper functions like setUser might have useful print statements :-)