search

cloudviz / agentless-system-crawler

A tool to crawl systems like crawlers for the web
Apache License 2.0
116 stars 44 forks source link

Test Mtgraphite emitter with new logmet endpoints #337

Open nadgowdas opened 6 years ago

nadgowdas commented 6 years ago

Description

As reported by @fabolive (https://github.ibm.com/watson-foundation-services/tracker/issues/9375#issuecomment-3496780), mtgraphite emitter works with legacy endpoints and does not work with new endpoints.

Need to test and fix emitter.

nadgowdas commented 6 years ago

I tried publishing metrics to the new logmet endpoint. I used metrics.stage1.ng.bluemix.net. Although, in crawler log it shows it has successfully 

2017-09-26 13:46:26,000 MainProcess INFO     Sent authentication with mtgraphite, returned length = 52
2017-09-26 13:46:26,262 MainProcess INFO     MTGraphite authentication server response of 1A
2017-09-26 13:46:26,364 MainProcess INFO     Confirmed write to mtgraphite socket.

on grafana, it does not show any data.

Another point, although crawler is started with frequency=60, i see only one emit log in the crawler o/p. It does not send logs periodically.

@fabolive @sahilsuneja1 @canturkisci

fabolive commented 6 years ago

@nadgowdas It seems to me that the new Logmet endpoints now can send "partial ACKs", that is, a message that contains fewer than the 6 bytes that comprise a full ACK. The behavior we are seeing in the mtgraphite emitter seems to be related to this change. For our reference, here is a pull request made against a nodeJS Logmet client I maintain to support the new endpoints: https://github.com/IBM/logmet-client-njs/pull/2

cc: @canturkisci @sahilsuneja1