Closed elisabettai closed 1 year ago
This is for node uuid: d0c14abe-f797-461f-aad4-99633fca92d6
, study uuid a369e1ea-73dc-11ed-881d-02420a0b00d4
. Something happened to the service on Dec 14, around 9:50 UTC. When it happened, no service/container was there anymore.
In Graylog, there's this (from dy_proxy):
{"level":"error","ts":1671011424.9446461,"logger":"http.log.error","msg":"dial tcp: lookup dy-sidecar-d0c14abe-f797-461f-aad4-99633fca92d6-0-jupyter-smash on 127.0.0.11:53: no such host","request":{"remote_addr":"172.13.81.13:47198","proto":"HTTP/1.1","method":"GET","host":"d0c14abe-f797-461f-aad4-99633fca92d6.services.osparc.io","uri":"/api/kernels/b509c5d1-d51a-47dc-b3d4-9f6cc214960c/channels?session_id=25234b82-bdde-4a88-b13e-b812f50ef595","headers":{"Pragma":["no-cache"],"Sec-Websocket-Extensions":["permessage-deflate; client_max_window_bits"],"Sec-Websocket-Version":["13"],"X-Forwarded-Host":["d0c14abe-f797-461f-aad4-99633fca92d6.services.osparc.io"],"X-Forwarded-Port":["443"],"X-Forwarded-Server":["manager1-production-simcore_production_traefik-1"],"Accept-Encoding":["gzip, deflate, br"],"Cookie":["_xsrf=2|8ab9c841|155130f0f1fd240572c24e72e2012382|1670592238"],"Connection":["Upgrade"],"Upgrade":["websocket"],"X-Forwarded-Proto":["https"],"X-Real-Ip":["195.176.8.125"],"Accept-Language":["en-US,en;q=0.9,de;q=0.8,fr;q=0.7"],"Origin":["https://d0c14abe-f797-461f-aad4-99633fca92d6.services.osparc.io"],"Sec-Websocket-Key":["VHjmLb+FqNDvGtXWCOLYcw=="],"Uber-Trace-Id":["1af2d7713449b63a:3d9e761a841b376c:2342f921e39e2de5:1"],"X-Forwarded-For":["195.176.8.125, 10.10.0.3"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"],"Cache-Control":["no-cache"]}},"duration":0.012051222,"status":502,"err_id":"aev88p3dy","err_trace":"reverseproxy.statusError (reverseproxy.go:858)"}
Then workspace and outputs were pushed right away. fyi @GitHK, in case there's something useful for you.
The query I run was from From: 2022-12-14 09:00:00.000
, Until: 2022-12-14 11:30:19.000
, search string: d0c14abe-f797-461f-aad4-99633fca92d6
(i.e. node uuid)
the misbehaving node
S4L Model Creation
just threw this error:_Issue created from a Mattermost message by
@newton
._