dispatcher: add metrics/observability and improve error handling

[fprasx]

ref https://neondb.slack.com/archives/C03TN5G758R/p1692714574250349

- [ ] https://github.com/neondatabase/autoscaling/pull/493
- [ ] https://github.com/neondatabase/autoscaling/pull/494
- [ ] https://github.com/neondatabase/autoscaling/pull/495
- [ ] https://github.com/neondatabase/autoscaling/pull/496
- [ ] https://github.com/neondatabase/autoscaling/pull/497
- [ ] https://github.com/neondatabase/autoscaling/pull/498
- [ ] https://github.com/neondatabase/autoscaling/pull/499
- [ ] https://github.com/neondatabase/autoscaling/pull/500

[sharnoff]

This was finished when #500 was merged; closing.