E2E TEST FAILURE: Elasticsearch pilot did not update the document count

[wallrj]

It looks to me like the Kubernetes couldn't create a PV for the ES pod.

• https://jetstack-build-infra.appspot.com/build/jetstack-logs/pr-logs/pull/jetstack_navigator/209/navigator-e2e-v1-8/623/

Seen in #209

==== START logs for container elasticsearch of pod test-elasticsearchcluster-1516805618-18656/es-test-mixed-0 ====
failed to get container status {"docker" "6265b4b7de6ce82b9f752e0e80aa493c67b374018274312307caae4ed5121638"}: rpc error: code = Unknown desc = Error: No such container: 6265b4b7de6ce82b9f752e0e80aa493c67b374018274312307caae4ed5121638==== END logs for container elasticsearch of pod test-elasticsearchcluster-1516805618-18656/es-test-mixed-0 ====
Cluster info dumped to standard output

• cluster-info dump

Jan 24 15:04:30 13471c29-0113-11e8-9b46-0a580a1c1f0d kubelet[3165]: E0124 15:04:30.295137    3165 kubelet.go:1628] Unable to mount volumes for pod "es-test-mixed-0_test-elasticsearchcluster-1516805618-18656(977edf87-0117-11e8-8830-525400dcf159)": timeout expired waiting for volumes to attach/mount for pod "test-elasticsearchcluster-1516805618-18656"/"es-test-mixed-0". list of unattached/unmounted volumes=[shared config elasticsearch-data es-test-token-d6xl6]; skipping pod
Jan 24 15:04:30 13471c29-0113-11e8-9b46-0a580a1c1f0d kubelet[3165]: E0124 15:04:30.295671    3165 pod_workers.go:182] Error syncing pod 977edf87-0117-11e8-8830-525400dcf159 ("es-test-mixed-0_test-elasticsearchcluster-1516805618-18656(977edf87-0117-11e8-8830-525400dcf159)"), skipping: timeout expired waiting for volumes to attach/mount for pod "test-elasticsearchcluster-1516805618-18656"/"es-test-mixed-0". list of unattached/unmounted volumes=[shared config elasticsearch-data es-test-token-d6xl6]
Jan 24 15:04:31 13471c29-0113-11e8-9b46-0a580a1c1f0d kubelet[3165]: I0124 15:04:31.285622    3165 kuberuntime_manager.go:500] Container {Name:elasticsearch Image:docker.elastic.co/elasticsearch/elasticsearch:5.6.2 Command:[/shared/pilot] Args:[--v=4 --logtostderr --pilot-name=$(POD_NAME) --pilot-namespace=$(POD_NAMESPACE) --elasticsearch-master-url=$(CLUSTER_URL) --elasticsearch-roles=$(ROLES) --elasticsearch-plugins=$(PLUGINS) --leader-election-config-map=$(LEADER_ELECTION_CONFIG_MAP)] WorkingDir: Ports:[{Name:transport HostPort:0 ContainerPort:9300 Protocol:TCP HostIP:} {Name:http HostPort:0 ContainerPort:9200 Protocol:TCP HostIP:}] EnvFrom:[] Env:[{Name:DISCOVERY_URL Value:es-test-discovery ValueFrom:nil} {Name:ROLES Value:master,ingest,data ValueFrom:nil} {Name:PLUGINS Value: ValueFrom:nil} {Name:LEADER_ELECTION_CONFIG_MAP Value:elastic-test-leaderelection ValueFrom:nil} {Name:CLUSTER_URL Value:http://es-test:9200 ValueFrom:nil} {Name:POD_NAME Value: ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.name,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,}} {Name:POD_NAMESPACE Value: ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,}}] Resources:{Limits:map[] Requests:map[cpu:{i:{value:500 scale:-3} d:{Dec:<nil>} s:500m Format:DecimalSI} memory:{i:{value:2147483648 scale:0} d:{Dec:<nil>} s:2Gi Format:BinarySI}]} VolumeMounts:[{Name:elasticsearch-data ReadOnly:false MountPath:/usr/share/elasticsearch/data SubPath: MountPropagation:<nil>} {Name:shared ReadOnly:true MountPath:/shared SubPath: MountPropagation:<nil>} {Name:config ReadOnly:false MountPath:/etc/pilot/elasticsearch/config SubPath: MountPropagation:<nil>} {Name:es-test-token-d6xl6 ReadOnly:true MountPath:/var/run/secrets/kubernetes.io/serviceaccount SubPath: MountPropagation:<nil>}] LivenessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/,Port:12000,Host:,Scheme:HTTP,HTTPHeaders:[],},TCPSocket:nil,},InitialDelaySeconds:60,TimeoutSeconds:5,PeriodSe
Jan 24 15:04:31 13471c29-0113-11e8-9b46-0a580a1c1f0d kubelet[3165]: conds:10,SuccessThreshold:1,FailureThreshold:3,} ReadinessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/,Port:12001,Host:,Scheme:HTTP,HTTPHeaders:[],},TCPSocket:nil,},InitialDelaySeconds:30,TimeoutSeconds:3,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,} Lifecycle:nil TerminationMessagePath:/dev/termination-log TerminationMessagePolicy:File ImagePullPolicy:IfNotPresent SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[IPC_LOCK],Drop:[],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,} Stdin:false StdinOnce:false TTY:false} is dead, but RestartPolicy says that we should restart it.

• minikube logs

/kind bug

[munnerz]

Seems like a minikube or kubernetes flake, based on volumes=[shared config elasticsearch-data es-test-token-d6xl6]; skipping pod.

It failed to even mount the service account secret into the pod. FWIW, we don't use PVs during tests (emptyDir is used).

I'm going to close this now as I don't think it is a bug in Navigator. Feel free to follow up and reopen if you disagree.

/close

[jetstack-bot]

@munnerz: The close prow plugin is deprecated, please migrate to the lifecycle plugin before April 2018

In response to [this](https://github.com/jetstack/navigator/issues/220#issuecomment-360421885): >Seems like a minikube or kubernetes flake, based on `volumes=[shared config elasticsearch-data es-test-token-d6xl6]; skipping pod`. > >It failed to even mount the service account secret into the pod. FWIW, we don't use PVs during tests (emptyDir is used). > >I'm going to close this now as I don't think it is a bug in Navigator. Feel free to follow up and reopen if you disagree. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/devel/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.