Hispter Shop failing on local Kubernetes Cluster

[hassanhamade]

Hi team, Trying to deploy hipster shop on a local Kubernetes cluster.

Currency and payment services in CrashLoopbackOff:

kubectl logs -f paymentservice-65bcb767c6-zcf94 @google-cloud/trace-agent ERROR StackdriverTracer#start: Tracing might not work as the following modules were loaded before the trace agent was initialized: [@google-cloud/common, @google-cloud/profiler, @google-cloud/projectify, @google-cloud/promisify, @protobufjs/aspromise, @protobufjs/base64, @protobufjs/eventemitter, @protobufjs/float, @protobufjs/inquire, @protobufjs/pool, @protobufjs/utf8, abbrev, agent-base, ansi-regex, aproba, are-we-there-yet, arrify, balanced-match, base64-js, bignumber.js, brace-expansion, buffer-equal-constant-time, code-point-at, concat-map, console-control-strings, console-log-level, core-util-is, debug, delay, delegates, detect-libc, duplexify, ecdsa-sig-formatter, end-of-stream, ent, es6-promisify, extend, findit2, fs.realpath, gauge, gaxios, gcp-metadata, glob, google-auth-library, gtoken, has-unicode, https-proxy-agent, inflight, inherits, is-fullwidth-code-point, isarray, json-bigint, jwa, jws, long, lru-cache, mime, minimatch, ms, node-fetch, node-pre-gyp, nopt, npmlog, number-is-nan, object-assign, once, os-homedir, os-tmpdir, osenv, p-limit, p-try, parse-duration, parse-ms, path-is-absolute, pify, pprof, pretty-ms, process-nextick-args, protobufjs, readable-stream, retry-request, rimraf, safe-buffer, semver, set-blocking, signal-exit, source-map, split, stream-shift, string-width, strip-ansi, teeny-request, through, through2, util-deprecate, uuid, wide-align, wrappy, yallist] D0917 23:37:12.992274043 1 env_linux.cc:71] Warning: insecure environment read function 'getenv' used

kubectl logs -f currencyservice-7f8658c69d-2xhg7 @google-cloud/trace-agent ERROR StackdriverTracer#start: Tracing might not work as the following modules were loaded before the trace agent was initialized: [@google-cloud/common, @google-cloud/profiler, @google-cloud/projectify, @google-cloud/promisify, @protobufjs/aspromise, @protobufjs/base64, @protobufjs/eventemitter, @protobufjs/float, @protobufjs/inquire, @protobufjs/pool, @protobufjs/utf8, abbrev, agent-base, ansi-regex, aproba, are-we-there-yet, arrify, balanced-match, base64-js, bignumber.js, brace-expansion, buffer-equal-constant-time, code-point-at, concat-map, console-control-strings, console-log-level, core-util-is, debug, delay, delegates, detect-libc, duplexify, ecdsa-sig-formatter, end-of-stream, ent, es6-promisify, extend, findit2, fs.realpath, gauge, gaxios, gcp-metadata, glob, google-auth-library, gtoken, has-unicode, https-proxy-agent, inflight, inherits, is-fullwidth-code-point, isarray, json-bigint, jwa, jws, long, lru-cache, mime, minimatch, ms, node-fetch, node-pre-gyp, nopt, npmlog, number-is-nan, object-assign, once, os-homedir, os-tmpdir, osenv, p-limit, p-try, parse-duration, parse-ms, path-is-absolute, pify, pprof, pretty-ms, process-nextick-args, protobufjs, readable-stream, retry-request, rimraf, safe-buffer, semver, set-blocking, signal-exit, source-map, split, stream-shift, string-width, strip-ansi, teeny-request, through, through2, util-deprecate, uuid, wide-align, wrappy, yallist] D0917 23:38:15.241807791 1 env_linux.cc:71] Warning: insecure environment read function 'getenv' used {"severity":"info","time":1568763495957,"message":"Starting gRPC server on port 7000...","pid":1,"hostname":"currencyservice-7f8658c69d-2xhg7","name":"currencyservice-server","v":1}

Seems to be an issue with the liveness and readiness probes:

Events: Type Reason Age From Message

---

Normal Scheduled 28m default-scheduler Successfully assigned default/currencyservice-7f8658c69d-2xhg7 to ddb3d636-12dc-4f86-9941-ddf3663f6b62 Normal Pulling 27m kubelet, ddb3d636-12dc-4f86-9941-ddf3663f6b62 pulling image "gcr.io/google-samples/microservices-demo/currencyservice:v0.1.2" Normal Pulled 27m kubelet, ddb3d636-12dc-4f86-9941-ddf3663f6b62 Successfully pulled image "gcr.io/google-samples/microservices-demo/currencyservice:v0.1.2" Warning Unhealthy 27m kubelet, ddb3d636-12dc-4f86-9941-ddf3663f6b62 Liveness probe failed: OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "read init-p: connection reset by peer": unknown Normal Created 26m (x4 over 27m) kubelet, ddb3d636-12dc-4f86-9941-ddf3663f6b62 Created container Normal Started 26m (x4 over 27m) kubelet, ddb3d636-12dc-4f86-9941-ddf3663f6b62 Started container Normal Pulled 26m (x3 over 27m) kubelet, ddb3d636-12dc-4f86-9941-ddf3663f6b62 Container image "gcr.io/google-samples/microservices-demo/currencyservice:v0.1.2" already present on machine Warning Unhealthy 26m (x2 over 27m) kubelet, ddb3d636-12dc-4f86-9941-ddf3663f6b62 Readiness probe failed: Warning Unhealthy 26m kubelet, ddb3d636-12dc-4f86-9941-ddf3663f6b62 Liveness probe failed: Warning BackOff 2m31s (x117 over 27m) kubelet, ddb3d636-12dc-4f86-9941-ddf3663f6b62 Back-off restarting failed container

Any idea how to work around this ?

[Freakin]

Seeing the exact same issue on my end with both these services. You ever figure it out?

[daniel-sanche]

what are you using for your local cluster? We typically test with kind, minikube, or Docker for Desktop

[mihara0320]

I'm having same issue too. In my case, using kubernetes from Docker for Desktop (windows)

[daniel-sanche]

Hmm I'm not able to reproduce this. Are others with this issue using Docker for Windows? I know Windows is less tested by our team. Are you using the pre-build manifests, or building locally?

[boyfzb2018]

I have the same issues on cartservice and loadgenerator services, my environment is K8S 1.17 with 1 Master,1 workNode

[daniel-sanche]

@boyfzb2018 Are you using Docker For Desktop Windows? Are you using the pre-build manifests, or building locally?

[boyfzb2018]

Thanks for your reply.The issue have fixed after solve flannel and network problem (Net traffic is blocked between Pods across Node)

发自我的iPhone

------------------ Original ------------------ From: Daniel Sanche <notifications@github.com> Date: Sat,Feb 29,2020 5:17 AM To: GoogleCloudPlatform/microservices-demo <microservices-demo@noreply.github.com> Cc: boyfzb2018 <173700751@qq.com>, Mention <mention@noreply.github.com> Subject: Re: [GoogleCloudPlatform/microservices-demo] Hispter Shop failing on local Kubernetes Cluster (#246)

@boyfzb2018 Are you using Docker For Desktop Windows? Are you using the pre-build manifests, or building locally?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.