filecoin-station / core

Station is a visionary network that connects your computer's idle resources to the Filecoin network and rewards you with FIL
https://filstation.app
Other
27 stars 8 forks source link

Noisy Sentry alerts from `lib/zinnia` #472

Open bajtos opened 4 months ago

bajtos commented 4 months ago

We are receiving too many new Sentry issues caused by the modules exiting. These issues are not actionable and create a lot of noise in our #alerts channel, which makes it difficult to spot real problems.

We should rework Station Core to report only meaningful & actionable problems to Sentry.

Tasks

Example errors

https://space-meridian.sentry.io/issues/5382499042/?referrer=slack&notification_uuid=f8c6ea8c-7274-44c9-aede-eda13dbbda80&alert_rule_id=15095305&alert_type=issue

Command failed with exit code 1: /tmp/.mount_filecoJaX6Ow/resources/core/modules/zinnia/zinniad voyager/main.js

[2024-05-20T23:12:30Z INFO  zinniad] Starting zinniad with config CliArgs { wallet_address: "0x499080263e9368c2b2a256635A9060463c484a49",...

https://space-meridian.sentry.io/issues/5382562354/?referrer=slack&notification_uuid=602dbae6-9ef8-4e88-bade-f6c07ef818c5&alert_rule_id=15095305&alert_type=issue

Command failed with exit code 1: /usr/src/app/modules/zinnia/zinniad voyager/main.js

[2024-05-20T23:35:41Z INFO  zinniad] Starting zinniad with config CliArgs { wallet_address: "0x8bF81E9648bE947474c5dE498B77FF0A001AA56F", station_id: "302a300506032...

https://space-meridian.sentry.io/issues/5382603361/?referrer=slack&notification_uuid=f09dc057-1731-4881-a110-96f519e754ab&alert_rule_id=15095305&alert_type=issue

Command failed with exit code 1: /tmp/.mount_filecoVMcfsm/resources/core/modules/zinnia/zinniad voyager/main.js

[2024-05-20T23:45:27Z INFO  zinniad] Starting zinniad with config CliArgs { wallet_address: "0xe6fC876219A081704AA5680AC5018c00D982E2aA",...

https://space-meridian.sentry.io/issues/5383121291/?referrer=slack&notification_uuid=3ba9438c-056f-4c69-9cba-12a404c04d97&alert_rule_id=15095305&alert_type=issue

Command failed with exit code 1: /usr/src/app/modules/zinnia/zinniad voyager/main.js

[2024-05-21T01:57:12Z INFO  zinniad] Starting zinniad with config CliArgs { wallet_address: "0x8bF81E9648bE947474c5dE498B77FF0A001AA56F", station_id: "302a300506032...

https://space-meridian.sentry.io/issues/5383447344/?referrer=slack&notification_uuid=7aeca151-406b-4f70-9945-517ebb810b8f&alert_rule_id=15095305&alert_type=issue

Command was canceled: '/Applications/Filecoin [Station.app/Contents/Resources/core/modules/zinnia/zinniad](http://station.app/Contents/Resources/core/modules/zinnia/zinniad)' voyager/main.js
The operation was aborted

[2024-05-21T03:05:12Z INFO  zinniad] Starting zinniad with config CliArgs { wallet_address: "0x72B509...

and so on and so on ...

bajtos commented 4 months ago

Screenshot of the Sentry Alerting rule - we can restore it after bringing down the number of false alerts.

https://space-meridian.sentry.io/alerts/rules/?project=4504792315199488&statsPeriod=14d

Screenshot 2024-05-21 at 09 09 54
juliangruber commented 4 months ago

I believe all of these are Voyager exits, and Voyager will be removed soon. Not sure we need to act on this, aside from ignoring these errors completely