Flakey contrib/mesos/pkg/executor test

[karlkfi]

Shippable failed (on PR https://github.com/GoogleCloudPlatform/kubernetes/pull/10613) durring a mesos executor unit test.

runtime: Go 1.3 env: KUBE_TEST_API_VERSIONS="v1beta3",KUBE_TEST_ETCD_PREFIXES="kubernetes.io/registry" cmd: KUBE_RACE="-race" KUBE_COVER="y" KUBE_GOVERALLS_BIN="$HOME/gopath/bin/goveralls" KUBE_TIMEOUT='-timeout 300s' KUBE_COVERPROCS=8 KUBE_TEST_ETCD_PREFIXES="${KUBE_TEST_ETCD_PREFIXES}" KUBE_TEST_API_VERSIONS="${KUBE_TEST_API_VERSIONS}" ./hack/test-go.sh -- -p=2

I0703 16:31:58.604976    8051 executor_test.go:134] resetting suicide watch with 1 task
I0703 16:31:58.605454    8051 executor_test.go:156] resetting suicide watch with 0 tasks
I0703 16:31:58.605681    8051 executor.go:429] resetting suicide watch timer for 1.5s
I0703 16:31:58.605750    8051 executor_test.go:178] resetting suicide watch with 1 task
I0703 16:31:58.605882    8051 executor.go:379] stopping suicide watch
I0703 16:31:58.605919    8051 executor_test.go:200] resetting suicide watch with 0 tasks
I0703 16:31:58.606043    8051 executor.go:390] resetting suicide watch
W0703 16:32:00.108825    8051 executor.go:369] Suicide timeout (1.5s) expired
W0703 16:32:00.108957    8051 executor_test.go:88] jumping?!
I0703 16:32:01.628734    8051 executor_test.go:223] jumps verified
I0703 16:32:01.638728    8051 executor.go:222] Executor nil of framework nil registered with slave nil
I0703 16:32:01.642248    8051 executor.go:222] Executor nil of framework nil registered with slave nil
I0703 16:32:01.642665    8051 executor.go:296] Slave is disconnected
I0703 16:32:01.646029    8051 executor.go:222] Executor nil of framework nil registered with slave nil
I0703 16:32:01.646380    8051 executor.go:296] Slave is disconnected
I0703 16:32:01.646564    8051 executor.go:245] Reregistered with slave nil
I0703 16:32:01.657187    8051 executor.go:222] Executor nil of framework nil registered with slave nil
I0703 16:32:01.701328    8051 executor.go:310] Launch task &TaskInfo{Name:*pod1.default.pods,TaskId:&TaskID{Value:*pod.0842a423-21a1-11e5-966b-0242ac114001,XXX_unrecognized:[],},SlaveId:&SlaveID{Value:*,XXX_unrecognized:[],},Resources:[&Resource{Name:*cpus,Type:*SCALAR,Scalar:&Value_Scalar{Value:*0,XXX_unrecognized:[],},Ranges:nil,Set:nil,Role:nil,XXX_unrecognized:[],} &Resource{Name:*mem,Type:*SCALAR,Scalar:&Value_Scalar{Value:*0,XXX_unrecognized:[],},Ranges:nil,Set:nil,Role:nil,XXX_unrecognized:[],}],Executor:&ExecutorInfo{ExecutorId:nil,FrameworkId:nil,Command:nil,Container:nil,Resources:[],Name:nil,Source:nil,Data:nil,XXX_unrecognized:[],},Command:nil,Container:nil,Data:*[123 34 107 105 110 100 34 58 34 80 111 100 34 44 34 97 112 105 86 101 114 115 105 111 110 34 58 34 118 49 98 101 116 97 51 34 44 34 109 101 116 97 100 97 116 97 34 58 123 34 110 97 109 101 34 58 34 112 111 100 49 34 44 34 110 97 109 101 115 112 97 99 101 34 58 34 100 101 102 97 117 108 116 34 44 34 115 101 108 102 76 105 110 107 34 58 34 47 97 112 105 47 118 49 98 101 116 97 51 47 112 111 100 115 47 92 117 48 48 48 49 34 44 34 99 114 101 97 116 105 111 110 84 105 109 101 115 116 97 109 112 34 58 110 117 108 108 125 44 34 115 112 101 99 34 58 123 34 99 111 110 116 97 105 110 101 114 115 34 58 91 123 34 110 97 109 101 34 58 34 102 111 111 34 44 34 105 109 97 103 101 34 58 34 34 44 34 112 111 114 116 115 34 58 91 123 34 99 111 110 116 97 105 110 101 114 80 111 114 116 34 58 56 48 48 49 44 34 112 114 111 116 111 99 111 108 34 58 34 84 67 80 34 125 93 44 34 114 101 115 111 117 114 99 101 115 34 58 123 125 44 34 99 97 112 97 98 105 108 105 116 105 101 115 34 58 123 125 125 93 125 44 34 115 116 97 116 117 115 34 58 123 34 67 111 110 100 105 116 105 111 110 34 58 91 123 34 116 121 112 101 34 58 34 82 101 97 100 121 34 44 34 115 116 97 116 117 115 34 58 34 84 114 117 101 34 125 93 125 125],HealthCheck:nil,XXX_unrecognized:[],}
I0703 16:32:01.741172    8051 executor.go:504] Binding 'default/pod1' to '' with annotations map[]...
I0703 16:32:02.753486    8051 executor.go:745] Kill task &TaskID{Value:*pod.0842a423-21a1-11e5-966b-0242ac114001,XXX_unrecognized:[],}
I0703 16:32:03.585784    8051 executor.go:222] Executor &ExecutorInfo{ExecutorId:&ExecutorID{Value:*ex1,XXX_unrecognized:[],},FrameworkId:nil,Command:&CommandInfo{Container:nil,Uris:[],Environment:nil,Shell:nil,Value:*k8sm-executor,Arguments:[],User:nil,XXX_unrecognized:[],},Container:nil,Resources:[],Name:nil,Source:nil,Data:*[80 75 3 4 20 0 8 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 115 112 111 100 46 106 115 111 110 92 144 49 107 196 48 12 133 103 231 87 4 207 57 46 165 75 233 92 232 26 40 116 41 55 40 177 122 39 234 216 198 214 29 13 37 255 189 82 146 107 160 75 130 63 233 61 61 222 79 101 44 36 122 199 92 40 6 251 92 219 219 131 109 4 126 81 112 250 236 162 91 222 35 50 56 96 16 38 26 99 3 140 168 243 146 162 59 144 59 180 139 204 88 15 61 250 162 91 245 223 206 103 140 182 169 237 224 175 133 49 43 233 33 219 122 174 204 172 214 37 225 112 183 29 98 96 160 32 113 132 124 40 250 119 106 187 99 44 141 112 94 176 167 62 67 158 142 225 76 225 123 27 166 152 121 117 168 119 203 78 160 176 167 182 217 163 93 152 147 36 57 173 50 79 55 12 88 74 151 99 143 91 36 193 24 160 247 168 117 112 190 98 179 66 158 210 238 176 49 10 196 4 254 5 61 76 111 40 135 157 134 120 108 183 177 174 190 34 175 229 36 224 139 234 143 90 77 186 71 211 82 140 209 207 124 170 244 63 255 6 0 0 255 255 80 75 7 8 234 120 48 10 245 0 0 0 162 1 0 0 80 75 3 4 20 0 8 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 115 112 111 100 50 46 106 115 111 110 92 144 65 107 195 48 12 133 207 206 175 8 62 167 52 219 46 99 231 193 174 129 193 46 163 7 37 214 90 49 199 54 182 90 22 70 254 251 164 38 93 160 151 4 127 122 122 122 188 223 202 88 72 244 129 185 80 12 246 165 182 151 7 219 8 252 166 224 244 217 69 119 125 143 200 224 128 65 152 236 24 27 96 68 157 151 20 221 142 220 174 125 84 153 177 30 122 244 69 85 245 191 230 43 70 219 212 118 240 231 194 152 149 244 144 109 61 87 102 86 235 146 112 184 217 14 49 48 80 144 56 66 62 21 221 157 90 239 24 75 35 28 175 216 83 159 33 79 251 112 164 240 179 14 83 204 188 56 212 155 101 39 80 216 115 219 108 209 78 204 73 146 28 150 53 79 23 12 88 74 151 99 143 107 36 193 24 160 247 168 117 112 62 99 179 64 158 210 230 176 50 10 196 4 254 21 61 76 239 40 135 157 134 120 106 215 177 74 223 144 151 114 18 240 73 247 247 90 77 186 69 211 82 140 209 207 124 168 244 63 255 5 0 0 255 255 80 75 7 8 145 39 93 195 246 0 0 0 162 1 0 0 80 75 3 4 20 0 8 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 13 0 0 0 100 105 114 47 115 112 111 100 46 106 115 111 110 92 144 65 107 195 48 12 133 207 206 175 8 62 167 52 163 151 177 243 96 215 192 96 151 209 131 18 107 173 152 99 27 91 45 11 35 255 125 82 147 46 208 75 130 63 61 61 61 222 111 101 44 36 250 192 92 40 6 251 82 219 235 147 109 4 126 83 112 250 236 162 187 189 71 100 112 192 32 76 118 140 13 48 162 206 75 138 110 71 110 215 30 84 102 172 135 30 125 81 85 253 175 249 138 209 54 181 29 252 165 48 102 37 61 100 91 207 149 153 213 186 36 28 238 182 67 12 12 20 36 142 144 79 69 15 167 214 59 198 210 8 167 27 246 212 103 200 211 62 156 40 252 172 195 20 51 47 14 245 102 217 9 20 246 220 54 91 180 51 115 146 36 199 101 205 211 21 3 150 210 229 216 227 26 73 48 6 232 61 106 29 156 47 216 44 144 167 180 57 172 140 2 49 129 127 69 15 211 59 202 97 167 33 14 237 58 86 233 27 242 82 78 2 62 235 254 94 171 73 247 104 90 138 49 250 153 143 149 254 231 191 0 0 0 255 255 80 75 7 8 135 16 169 50 246 0 0 0 162 1 0 0 80 75 1 2 20 0 20 0 8 0 8 0 0 0 0 0 234 120 48 10 245 0 0 0 162 1 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 115 112 111 100 46 106 115 111 110 80 75 1 2 20 0 20 0 8 0 8 0 0 0 0 0 145 39 93 195 246 0 0 0 162 1 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 44 1 0 0 115 112 111 100 50 46 106 115 111 110 80 75 1 2 20 0 20 0 8 0 8 0 0 0 0 0 135 16 169 50 246 0 0 0 162 1 0 0 13 0 0 0 0 0 0 0 0 0 0 0 0 0 90 2 0 0 100 105 114 47 115 112 111 100 46 106 115 111 110 80 75 5 6 0 0 0 0 3 0 3 0 170 0 0 0 139 3 0 0 0 0],XXX_unrecognized:[],} of framework nil registered with slave nil
E0703 16:32:09.626756    8051 file.go:53] Unable to read config path "/tmp/executor-k8sm-archive703888931": path does not exist, ignoring
--- FAIL: TestExecutorStaticPods (6.87 seconds)
    executor_test.go:530: Executor should send pod updates for 2 pods, only saw 0
I0703 16:32:09.635685    8051 executor.go:222] Executor nil of framework nil registered with slave nil
I0703 16:32:09.645881    8051 executor.go:813] Receives message from framework test framework message
I0703 16:32:09.647379    8051 executor.go:310] Launch task &TaskInfo{Name:*pod1.default.pods,TaskId:&TaskID{Value:*foo,XXX_unrecognized:[],},SlaveId:&SlaveID{Value:*,XXX_unrecognized:[],},Resources:[&Resource{Name:*cpus,Type:*SCALAR,Scalar:&Value_Scalar{Value:*0,XXX_unrecognized:[],},Ranges:nil,Set:nil,Role:nil,XXX_unrecognized:[],} &Resource{Name:*mem,Type:*SCALAR,Scalar:&Value_Scalar{Value:*0,XXX_unrecognized:[],},Ranges:nil,Set:nil,Role:nil,XXX_unrecognized:[],}],Executor:&ExecutorInfo{ExecutorId:nil,FrameworkId:nil,Command:nil,Container:nil,Resources:[],Name:nil,Source:nil,Data:nil,XXX_unrecognized:[],},Command:nil,Container:nil,Data:*[123 34 107 105 110 100 34 58 34 80 111 100 34 44 34 97 112 105 86 101 114 115 105 111 110 34 58 34 118 49 98 101 116 97 51 34 44 34 109 101 116 97 100 97 116 97 34 58 123 34 110 97 109 101 34 58 34 112 111 100 49 34 44 34 110 97 109 101 115 112 97 99 101 34 58 34 100 101 102 97 117 108 116 34 44 34 115 101 108 102 76 105 110 107 34 58 34 47 97 112 105 47 118 49 98 101 116 97 51 47 112 111 100 115 47 92 117 48 48 48 49 34 44 34 99 114 101 97 116 105 111 110 84 105 109 101 115 116 97 109 112 34 58 110 117 108 108 125 44 34 115 112 101 99 34 58 123 34 99 111 110 116 97 105 110 101 114 115 34 58 91 123 34 110 97 109 101 34 58 34 102 111 111 34 44 34 105 109 97 103 101 34 58 34 34 44 34 112 111 114 116 115 34 58 91 123 34 99 111 110 116 97 105 110 101 114 80 111 114 116 34 58 56 48 48 49 44 34 112 114 111 116 111 99 111 108 34 58 34 84 67 80 34 125 93 44 34 114 101 115 111 117 114 99 101 115 34 58 123 125 44 34 99 97 112 97 98 105 108 105 116 105 101 115 34 58 123 125 125 93 125 44 34 115 116 97 116 117 115 34 58 123 34 67 111 110 100 105 116 105 111 110 34 58 91 123 34 116 121 112 101 34 58 34 82 101 97 100 121 34 44 34 115 116 97 116 117 115 34 58 34 84 114 117 101 34 125 93 125 125],HealthCheck:nil,XXX_unrecognized:[],}
I0703 16:32:09.666072    8051 executor.go:504] Binding 'default/pod1' to '' with annotations map[]...
E0703 16:32:10.636943    8051 file.go:53] Unable to read config path "/tmp/executor-k8sm-archive703888931": path does not exist, ignoring
I0703 16:32:10.677556    8051 executor.go:813] Receives message from framework task-lost:foo
I0703 16:32:10.678014    8051 executor.go:813] Receives message from framework kamikaze
I0703 16:32:10.678086    8051 executor.go:466] Attempting suicide
I0703 16:32:10.678174    8051 executor.go:868] Stopping executor driver
I0703 16:32:10.678235    8051 executor.go:876] Shutdown the executor
I0703 16:32:10.678293    8051 executor.go:890] exiting
E0703 16:32:10.678350    8051 executor.go:846] exiting with unclean shutdown: <nil>
I0703 16:32:10.680084    8051 executor.go:222] Executor nil of framework nil registered with slave nil
I0703 16:32:10.680235    8051 executor.go:868] Stopping executor driver
I0703 16:32:10.680308    8051 executor.go:876] Shutdown the executor
I0703 16:32:10.680369    8051 executor.go:890] exiting
E0703 16:32:10.680576    8051 executor.go:846] exiting with unclean shutdown: <nil>
I0703 16:32:10.681876    8051 executor.go:222] Executor nil of framework nil registered with slave nil
FAIL
coverage: 70.9% of statements
FAIL    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/executor    12.336s

[sttts]

@karlkfi: Do you know which git sha1 this happened with? I looked at the executor code printing out

I0703 16:32:09.635685    8051 executor.go:222] Executor nil of framework nil registered with slave nil

With the current code in master the first nil seems to be impossible.