[Bug] json_exporter export jvm and grpc metrics to promethus format happen error

[smlHao]

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

Search before asking

• [X] I have searched in the issues and found no similar issues.

Describe the bug

json_exporter export jvm and grpc metrics to promethus format happen error

json_exporter config.yaml : modules: default: headers: MyHeader: MyHeaderValue metrics:

• name: rss_shuffle_server type: object help: General statistics of the top cryptocurrencies path: '{.metrics[*]}' labels: name: 'rssworker{.name}' values: value1: '{.value}'

start json_exporter: ./json_exporter --config.file config.yml

then call : curl http://localhost:7979/probe?target=http://[uniffle serverIp]:[jettyPort]/metrics/jvm , error happend:

guess the error happend because of the metrics value like "value":3.3737474048E10 , "value":"NaN"

Affects Version(s)

0.7.1

Uniffle Server Log Output

No response

Uniffle Engine Log Output

No response

Uniffle Server Configurations

No response

Uniffle Engine Configurations

No response

Additional context

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

[jerqi]

The value of 3.3737474048E10 is correct. The value use scientific notation.

[smlHao]

The value of 3.3737474048E10 is correct. The value use scientific notation.

Thank your fro fast reply !!!! yes, I know，I think json_exporter has no handler to match this, /metrics/server works fine, so is it possible keep same with /metrics/serve , don`t use scientific notation?

in addition，/metrics/grpc has "value":"NaN"

[jerqi]

You should use prometheus api like /prometheus/metrics/server instead of /metrics/server.

[smlHao]

ok，thank you，because metrics is json format and had deployed  promethus，so i chose json_exporter to pull the metrics send to promethus ; if use promethus api instead of uniffle metrics api，do you have some documents for doing this?

---Original--- From: @.> Date: Wed, Jul 5, 2023 18:58 PM To: @.>; Cc: @.**@.>; Subject: Re: [apache/incubator-uniffle] [Bug] json_exporter export jvm andgrpc metrics to promethus format happen error (Issue #996)

You should use prometheus api like /prometheus/metrics/server instead of /metrics/server.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[jerqi]

May this is useful for you. https://github.com/apache/incubator-uniffle/blob/master/docs/server_guide.md#prometheuspushgatewaymetricreporter-settings

[smlHao]

OK ，thank you very much!    I  will  try it

---Original--- From: @.> Date: Wed, Jul 5, 2023 19:24 PM To: @.>; Cc: @.**@.>; Subject: Re: [apache/incubator-uniffle] [Bug] json_exporter export jvm andgrpc metrics to promethus format happen error (Issue #996)

May this is useful for you. https://github.com/apache/incubator-uniffle/blob/master/docs/server_guide.md#prometheuspushgatewaymetricreporter-settings

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[smlHao]

May this is useful for you. https://github.com/apache/incubator-uniffle/blob/master/docs/server_guide.md#prometheuspushgatewaymetricreporter-settings

Hi, I had try it and it works fine,the metrics had been imported into promethus, now I want to display them in grafana, do you have a grafana dashboard json which can be imported in grafana? if so, this can help us easy to monitor. Thanks

[jerqi]

Hi, I had try it and it works fine,the metrics had been imported into promethus, now I want to display them in grafana, do you have a grafana dashboard json which can be imported in grafana? if so, this can help us easy to monitor. Thanks

I don't have. Maybe @xianjingfeng has the grafana dashboard json.

[smlHao]

@xianjingfeng Hi, do you have the grafana dashboard json file?

[xianjingfeng]

@xianjingfeng Hi, do you have the grafana dashboard json file?

https://docs.google.com/document/d/1xKblAIoRgpwf3iTPTM1rq_wOJH4dIhnM2Skx56fMgMM/edit?usp=sharing

[smlHao]

@xianjingfeng Hi, do you have the grafana dashboard json file?

https://docs.google.com/document/d/1xKblAIoRgpwf3iTPTM1rq_wOJH4dIhnM2Skx56fMgMM/edit?usp=sharing

Hi, got it, Thanks !!!

[xianjingfeng]

@xianjingfeng Hi, do you have the grafana dashboard json file?

https://docs.google.com/document/d/1xKblAIoRgpwf3iTPTM1rq_wOJH4dIhnM2Skx56fMgMM/edit?usp=sharing

Hi, got it, Thanks !!!

@xianjingfeng Hi,I had try it and find this grafana json file is used for coordinator cluster, is there another file grafana json file for shuffle server? if so,can you share me? Thanks !!!

Already included shuffle server.

[smlHao]

@xianjingfeng Hi, do you have the grafana dashboard json file?

https://docs.google.com/document/d/1xKblAIoRgpwf3iTPTM1rq_wOJH4dIhnM2Skx56fMgMM/edit?usp=sharing

Hi, got it, Thanks !!!

@xianjingfeng Hi,I had try it and find this grafana json file is used for coordinator cluster, is there another file grafana json file for shuffle server? if so,can you share me? Thanks !!!

Already included shuffle server.

yes, I had known about this , Thanks !!!

[advancedxy]

https://docs.google.com/document/d/1xKblAIoRgpwf3iTPTM1rq_wOJH4dIhnM2Skx56fMgMM/edit?usp=sharing

Hi, @xianjingfeng do you think it's possible to include this grafana json file as a template file in the uniffle repo and update the doc to include that part? It might be useful.

[xianjingfeng]

https://docs.google.com/document/d/1xKblAIoRgpwf3iTPTM1rq_wOJH4dIhnM2Skx56fMgMM/edit?usp=sharing

Hi, @xianjingfeng do you think it's possible to include this grafana json file as a template file in the uniffle repo and update the doc to include that part? It might be useful.

Yes, just like ambari do. https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/common-services/AMBARI_METRICS/3.0.0/package/files/grafana-dashboards/default But I think the file I gave is not good enough, like a draft.

[jerqi]

Solved.