AutohomeCorp / frostmourne

Frostmourne(霜之哀伤监控平台)是基于Elasticsearch, Prometheus, SkyWalking, InfluxDB,Mysql/TiDB,ClickHouse, SqlServer, IoTDB数据的分布式监控报警系统. Monitor & alert & alarm for Elasticsearch,Prometheus data。主要使用springboot2 + vue-element-admin
https://frostmourne-demo.github.io
MIT License
1.41k stars 314 forks source link

配置报警升级不生效, 报如下的错, 麻烦帮忙看看 #168

Open codech opened 5 months ago

codech commented 5 months ago

"2024-06-21T15:43:19.011+08:00" "ERROR" "com.autohome.frostmourne.monitor.service.core.execute.AlarmExecutor" "51" "java.lang.RuntimeException" "error when totalCount" "error when doRule" "com.autohome.frostmourne.monitor.service.core.execute.AlarmExecutor" "doRule" "-" "-" "tech" "frostmourne" "frostmourne.monitor" "-" "225" "http-nio-10054-exec-9" "-" "172.21.48.19" "-" "-" "-" "-" "java.lang.RuntimeException: error when totalCount at com.autohome.frostmourne.monitor.dao.elasticsearch.EsRestClientContainer.queryElasticsearchMetricValue(EsRestClientContainer.java:215) at com.autohome.frostmourne.monitor.service.core.query.impl.ElasticsearchDataQuery.queryElasticsearchMetricValue(ElasticsearchDataQuery.java:57) at com.autohome.frostmourne.monitor.service.core.metric.elasticsearch.ElasticsearchNumericMetric.pullMetricData(ElasticsearchNumericMetric.java:26) at com.autohome.frostmourne.monitor.service.core.metric.AbstractNumericMetric.pullMetric(AbstractNumericMetric.java:33) at com.autohome.frostmourne.monitor.service.core.rule.AbstractRule.context(AbstractRule.java:42) at com.autohome.frostmourne.monitor.service.core.rule.NumericRule.verify(NumericRule.java:19) at com.autohome.frostmourne.monitor.service.core.execute.AlarmExecutor.doRule(AlarmExecutor.java:47) at com.autohome.frostmourne.monitor.service.core.execute.AlarmExecutor.execute(AlarmExecutor.java:37) at com.autohome.frostmourne.monitor.service.core.execute.AlarmService.run(AlarmService.java:69) at com.autohome.frostmourne.monitor.service.core.execute.AlarmService.run(AlarmService.java:49) at com.autohome.frostmourne.monitor.controller.AlarmController.run(AlarmController.java:45) at sun.reflect.GeneratedMethodAccessor278.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)

codech commented 5 months ago

也不是不生效, 刚开始是生效的, 过了一天之后 就不往升级后的webhook发了

sanutey commented 4 months ago

从日志看像是查询 ES 时报错了,看下 ES 是正常的吗

codech commented 4 months ago

ES是正常的