search

helium / miner

Miner for the helium blockchain
Apache License 2.0
609 stars 266 forks source link

Gather logs from helium/miner software for performance/critical error monitoring #1241

Open wolasss opened 2 years ago

wolasss commented 2 years ago

Monitoring the health of the blockchain and p2p network is pretty challenging. I am a software developer myself, I am following Helium for quite some time and I think there is room for improvement here and I have suggestion that could be beneficial for Helium Core team.

Did you consider using a tool for gathering logs in one place from the miners to monitor issues that are happening in the miner's software?

I have used Sentry in the past, and it turned to be very useful to monitor what is going on in the code released to production. By using this tool, I have tracked errors that were very hard to catch (some nasty edge cases I wouldn't think of as of writing the code) but also if there were any regression you could immediately see the reflection in the logs and track the cause quickly. Also, a tool of this kind gives you very useful statistics for tracking the errors and their context in general.

I think Helium could really benefit from that. Otherwise, you have to wait for the users to report the issues (and please note, that most manfuacturers don't provide helium/miner logs for the end-user) and sometimes this may not even happen, cause not every end-user know what to look for and where.

I feel like right now you realize something is wrong in the network when people complain about no witness/no rewards. And there are probably many more errors that happen in the miner's software that happen in some particular context (like https://github.com/helium/miner/issues/1239 for instance).

The tool I mentioned - Sentry is a paid (SaaS) when hosted in the cloud, but you can run your own standalone instance in your own infrastructure. I think there is an erlang library to use the Sentry API https://github.com/artemeff/raven-erlang. But I am pretty sure there are other similar solutions for error tracking that could be more suitable

boozzard commented 2 years ago

Yes or even the ability to send syslog out both from the application and the underlying OS.

etotour commented 2 years ago

Hello, We need technical equipment 100 - 200 pcs. who can send it to Russia (SPB MSK SEBIRIAN SOCHI) / Bulgary / Georgia / Armenia / Tatarstan / .. ? better to Russia, then.. in the future we ourselves.. How long will it take for a technician, and is it possible even more volume than 200 pcs? Regards, etotour telegram: +7 916 989 96 36 etotour@gmail.com