Closed fainashalts closed 1 month ago
tremarkley
commented
2 months ago Created a mixpanel dashboard for tracking usage of faucet on dev console: https://mixpanel.com/s/2Y71kt
Based on the usage I am seeing it doesnt seem like the captcha has had any effect on the number of drips. I still suspect there are people botting this based on the number of users using Privy auth and also the fact that so many users are from one region:
tremarkley
commented
2 months ago Started working on updating the grafana dashboard. The "Count of faucet drip failures" panel has been updated to be pointed at the faucet failures on dev console.
The drippie contract balance panel is pointed to the right contract.
The L1 Sepolia Faucet Contract and L1 Sepolia Faucet Admin balance panels are still pointed to the old faucet, but should automatically update to the new faucet once this merges and the gateway pods are updated: https://github.com/ethereum-optimism/k8s/pull/4283
Next steps would be to add some alerting so that we get alerted when drip failures increase.
tremarkley
commented
1 month ago Created 4 grafana alerts for the faucet:
For now these alerts will ping in the #pod-devx slack channel. Next step is getting the alerts integrated into ops genie so that the alerts get sent to the on call engineer.
If you're curious, the alerts can be found here: https://optimistic.grafana.net/alerting/list?search=dashboard:ebc95cfa-3368-410f-9d72-ca240f4e2831
Add observability for status of faucet requests to get a sense for how often folks encounter failures. Most likely should use the existing Grafana dashboard. Once this task is complete, we should add a pageable event at a certain threshold (we can experiment with the right level) for oncall to ensure we are alerted.