bahamat
commented
4 years ago @tangfeng123 There have been a lot of updates since this ticket was filed, and we've never seen this error. If it's still an issue, please re-open this ticket.
If it is still happening, it would be helpful if you can find the log message from when the CN is added to the cache, rather than when the client attempts to fetch metrics. If you're not able to find it, taking a core file (gcore <process id>) and sending it to us might help identify what's wrong.
When i try get metrics like below it give me the error code 500 like below. curl --insecure --cert-type pem --cert promcert.pem --key promkey.pem "https://191c0d4f-5bd4-e611-f673-c1b799de577b.cmon.bst-1.cns.bstjpc.com:9163/metrics" {"code":"InternalError","message":"Internal error"}
Then I login cmon. and check the log, it is show that error message admin ip cache obj (object) is required. the log like below, there have 6 servers(cn) in my env, just one cn's instances have this problem, Please help. thank you
{"name":"cmon","hostname":"e8c5201b-e843-4a08-a3d9-a4f45833cab3","pid":20211,"req_id":"22329df3-aa5a-4d85-8878-78a8728be763","level":50,"err":{"message":"admin ip cache obj (object) is required","name":"AssertionError","stack":"AssertionError: admin ip cache obj (object) is required\n at Server.apiGetMetrics (/opt/triton/cmon/lib/endpoints/metrics.js:37:20)\n at next (/opt/triton/cmon/node_modules/restify/lib/server.js:906:30)\n at f (/opt/triton/cmon/node_modules/restify/node_modules/once/once.js:25:25)\n at Server.rateLimit (/opt/triton/cmon/node_modules/restify/lib/plugins/throttle.js:266:21)\n at next (/opt/triton/cmon/node_modules/restify/lib/server.js:906:30)\n at f (/opt/triton/cmon/node_modules/restify/node_modules/once/once.js:25:25)\n at Server.authorizationHandler (/opt/triton/cmon/lib/common.js:88:13)\n at next (/opt/triton/cmon/node_modules/restify/lib/server.js:906:30)\n at f (/opt/triton/cmon/node_modules/restify/node_modules/once/once.js:25:25)\n at handleMahiAccount (/opt/triton/cmon/lib/common.js:47:21)"},"route":"getmetrics","req":{"method":"GET","url":"/metrics","headers":{"host":"191c0d4f-5bd4-e611-f673-c1b799de577b.cmon.bst-1.cns.bstjpc.com:9163","user-agent":"Go-http-client/1.1","accept":"application/vnd.google.protobuf;proto=io.prometheus.client.MetricFamily;encoding=delimited;q=0.7,text/plain;version=0.0.4;q=0.3,/;q=0.1","accept-encoding":"gzip","connection":"close"},"remoteAddress":"192.168.9.68","remotePort":42518},"msg":"Uncaught exception","time":"2017-03-29T07:17:31.168Z","v":0}