search

appclacks / mirabelle

A stream processing engine for monitoring
Eclipse Public License 1.0
95 stars 5 forks source link

Connecting to Grafana Failed #11

Closed stephenjones5543 closed 2 years ago

stephenjones5543 commented 3 years ago

Hello,

I am using a docker image to deploy Mirabelle in a kubernetes cluster, I wrote a custom helm chart to manage my deployment, I was able to set everything up until connecting to Grafana with the websocket ws://localhost:5556

I am using riemann Grafana plugin Mirabelle version V0.4.0 Grafana version v8.1.0

Please see error below.

{"@timestamp":"2021-08-12T18:45:49.927Z","@version":"1","message":"Error in the websocket handler","logger_name":"mirabelle.transport.websocket","thread_name":"worker-3","level":"ERROR","level_value":40000,"stack_trace":"java.lang.NullPointerException: null\n\tat java.base/java.util.regex.Matcher.getTextLength(Matcher.java:1770)\n\tat java.base/java.util.regex.Matcher.reset(Matcher.java:416)\n\tat java.base/java.util.regex.Matcher.<init>(Matcher.java:253)\n\tat java.base/java.util.regex.Pattern.matcher(Pattern.java:1133)\n\tat java.base/java.util.regex.Pattern.split(Pattern.java:1261)\n\tat java.base/java.util.regex.Pattern.split(Pattern.java:1334)\n\tat clojure.string$split.invokeStatic(string.clj:224)\n\tat clojure.string$split.invoke(string.clj:219)\n\tat mirabelle.transport.websocket$http_query_map.invokeStatic(websocket.clj:24)\n\tat mirabelle.transport.websocket$http_query_map.invoke(websocket.clj:18)\n\tat mirabelle.transport.websocket$handler$handle__21125$fn__21126.invoke(websocket.clj:83)\n\tat mirabelle.transport.websocket$handler$handle__21125.invoke(websocket.clj:79)\n\tat org.httpkit.server.HttpHandler.run(RingHandler.java:117)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:834)\n"

Thanks

mcorbin commented 3 years ago

Hello, Thank you for testing Mirabelle.

The Grafana plugin works with Mirabelle but has been designed for Riemann. Mirabelle and Riemann do not use the same query language for the index (https://www.mirabelle.mcorbin.fr/howto/index/).

For example, I make it work with this configuration:

mirabelle-grafana

The query is WzphbHdheXMtdHJ1ZV0= which is [:always-true] (match all events) in base64. Ideally, a Mirabelle grafana plugin would automatically convert the query to base64 (the Riemann plugin could be extended).

stephenjones5543 commented 3 years ago

Hey,

Thanks for the swift response. I feel the error occurred when establishing the web socket connection

{"@timestamp":"2021-08-13T21:46:17.866Z","@version":"1","message":"Error in the websocket handler","logger_name":"mirabelle.transport.websocket","thread_name":"worker-4","level":"ERROR","level_value":40000,

Could it be the handler?

mcorbin commented 3 years ago

Thanks for the swift response. I feel the error occurred when establishing the web socket connection

Which query do you use in Grafana ?

faxm0dem commented 2 years ago

I'm getting the same Error in the websocket handler when using the query true. I realise this isn't a valid query, and when using your example base64 string it works as expected. Proposed change: maybe send a more intelligible error message back to user, e.g. Parse error in query string "true". Please check the syntax. Docs here : ...

faxm0dem commented 2 years ago

For the record here is the full error message on the server side:

{
  "@timestamp": "2021-10-19T09:33:15.237+02:00",
  "@version": "1",
  "message": "Error in the websocket handler",
  "logger_name": "mirabelle.transport.websocket",
  "thread_name": "worker-3",
  "level": "ERROR",
  "level_value": 40000,
  "stack_trace": "clojure.lang.ExceptionInfo: Invalid condition
        at exoscale.ex$ex_info.invokeStatic(ex.cljc:273)
        at exoscale.ex$ex_info.invoke(ex.cljc:253)
        at exoscale.ex$ex_incorrect.invokeStatic(ex.cljc:316)
        at exoscale.ex$ex_incorrect.invoke(ex.cljc:316)
        at exoscale.ex$ex_incorrect_BANG_.invokeStatic(ex.cljc:316)
        at exoscale.ex$ex_incorrect_BANG_.invoke(ex.cljc:316)
        at mirabelle.action.condition$compile_condition.invokeStatic(condition.clj:48)
        at mirabelle.action.condition$compile_condition.invoke(condition.clj:40)
        at mirabelle.action.condition$fn__4758.invokeStatic(condition.clj:74)
        at mirabelle.action.condition$fn__4758.invoke(condition.clj:57)
        at clojure.lang.AFn.applyToHelper(AFn.java:154)
        at clojure.lang.AFn.applyTo(AFn.java:144)
        at clojure.core$apply.invokeStatic(core.clj:667)
        at clojure.core$memoize$fn__6894.doInvoke(core.clj:6342)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at mirabelle.transport.websocket$request__GT_pred.invokeStatic(websocket.clj:36)
        at mirabelle.transport.websocket$request__GT_pred.invoke(websocket.clj:33)
        at mirabelle.transport.websocket$handler$handle__26079$fn__26080.invoke(websocket.clj:85)
        at mirabelle.transport.websocket$handler$handle__26079.invoke(websocket.clj:79)
        at org.httpkit.server.HttpHandler.run(RingHandler.java:117)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
"
}
mcorbin commented 2 years ago

Queries should indeed be in base64 but the query true also works (I made a special check for it):

query

mcorbin commented 2 years ago

I improved the error message. it's still a stack trace but now it says that the websocket query is invalid:

@timestamp":"2021-10-22T21:08:27.471+02:00","@version":"1","message":"Error in the websocket handler","logger_name":"mirabelle.transport.websocket","thread_name":"worker-1","level":"ERROR","level_value":40000,"stack_trace":"clojure.lang.ExceptionInfo: Invalid websocket query azezaea
\tat exoscale.ex$ex_info.invokeStatic(ex.cljc:273)
\tat exoscale.ex$ex_info.invoke(ex.cljc:253)
\tat exoscale.ex$ex_incorrect.invokeStatic(NO_SOURCE_FILE:0)
\tat exoscale.ex$ex_incorrect.invoke(NO_SOURCE_FILE:0)
\tat exoscale.ex$ex_incorrect_BANG_.invokeStatic(NO_SOURCE_FILE:0)
\tat exoscale.ex$ex_incorrect_BANG_.invoke(NO_SOURCE_FILE:0)
\tat mirabelle.transport.websocket$request__GT_pred.invokeStatic(websocket.clj:45)
\tat mirabelle.transport.websocket$request__GT_pred.invoke(websocket.clj:34)
\tat mirabelle.transport.websocket$handler$handle__21578$fn__21579.invoke(websocket.clj:91)
\tat mirabelle.transport.websocket$handler$handle__21578.invoke(websocket.clj:85)
\tat org.httpkit.server.HttpHandler.run(RingHandler.java:117)
\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
\tat java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol
\tat clojure.lang.RT.seqFrom(RT.java:557)
\tat clojure.lang.RT.seq(RT.java:537)
\tat clojure.lang.RT.first(RT.java:693)
\tat clojure.core$first__5401.invokeStatic(core.clj:55)
\tat clojure.core$first__5401.invoke(core.clj:55)
\tat mirabelle.action.condition$fn__1537.invokeStatic(condition.clj:65)
\tat mirabelle.action.condition$fn__1537.invoke(condition.clj:57)
\tat clojure.lang.AFn.applyToHelper(AFn.java:154)
\tat clojure.lang.AFn.applyTo(AFn.java:144)
\tat clojure.core$apply.invokeStatic(core.clj:667)
\tat clojure.core$memoize$fn__6894.doInvoke(core.clj:6342)
\tat clojure.lang.RestFn.invoke(RestFn.java:408)
\tat mirabelle.transport.websocket$request__GT_pred.invokeStatic(websocket.clj:39)
\t... 9 common frames omitted
mcorbin commented 2 years ago

Closing this issue (unable to reproduce the original bug), please reopen with more information if needed.