search

threefoldtech / farmerbot

ability to manage a farm
Apache License 2.0
4 stars 1 forks source link

Enable monitoring of farmerbot activity through RMB job #15

Open Jelleeelco opened 1 year ago

Jelleeelco commented 1 year ago

Occasionally the ZOS image doesn't download completely on a node during the periodic wakeup. This causes the node to be unavailable. The farmerbot will then stop shutting down any other node (this makes sense).

It would be very helpful to get some monitoring tools on the activity of the farmerbot. Like automated status report via email or telegram bot. Something like this:

scottyeager commented 1 year ago

The node status bot can be adapted to provide this kind of info for farmerbot controlled nodes. I'll be working on this as part of a larger update to support changes in 3.9.

brandonpille commented 1 year ago

What is the node status bot? Because I received a request from someone from the community asking for a way to get information on what is happening in the farmerbot at any time. So I was thinking of adding a rest api or a websocket messaging service that will expose the events from the farmerbot.

Jelleeelco commented 1 year ago

Hi Brandon, The status bot is a telegram bot that notifies when a node is down, Scott is the developer behind it (I believe it notifies only when both ping and dashboard are down). One can also ping the nodes, to check if they are online. Now with the farmerbot, ping won't work and the dashboard will show nodes down all the time, hence the status bot needs to be changed.

a rest api / websocket msg service would be ideal of course!

brandonpille commented 1 year ago

Ok. I'll create an issue for it so that. @scottyeager what should the rest api / websocket msg service be able to do?

scottyeager commented 1 year ago

Hey @brandonpille, just catching up with this. My plan for the node status bot was to use publicly available data from TF Chain along with pings over RMB to determine if nodes are waking up when they should. Also having some data coming directly from the bot could be helpful.

Since the bot is a service I operate that people only interact with through Telegram, it wouldn't be able to reach a web API for farmerbots that aren't exposed to the internet. Offering some RMB endpoints to check on the farmerbot would make more sense for this use case.

Here are a few ideas:

brandonpille commented 1 year ago

Hey @brandonpille, just catching up with this. My plan for the node status bot was to use publicly available data from TF Chain along with pings over RMB to determine if nodes are waking up when they should. Also having some data coming directly from the bot could be helpful.

Since the bot is a service I operate that people only interact with through Telegram, it wouldn't be able to reach a web API for farmerbots that aren't exposed to the internet. Offering some RMB endpoints to check on the farmerbot would make more sense for this use case.

Here are a few ideas:

  • Simple ping to see that the farmerbot is online
  • Return the scheduled wake up time for this farmerbot
  • Info on nodes failing to wake up as scheduled
  • Retrieve the last power target change and its time

Opening that up via RMB would make a lot of sense indeed. That would even require almost no change to the code. So I'll go ahead and do that.

ashraffouda commented 8 months ago

blocked because of https://github.com/threefoldtech/farmerbot/issues/67#issuecomment-1790639954