Automatic client upgrades

Crate

Summary

Add support in Hermes to detect a chain upgrade and to automatically trigger the upgrading of all clients which target the upgraded chain.

Problem Definition

The problem is that currently Hermes has only a manual upgrade client command, and we are at a WIP stage on adding a bulk upgrade clients CLI (#763). It would be good to support client upgrading in a reactive manner; so this feature is an extension to the reactive capabilities (#823) of Hermes, to additionally support client upgrades.

Another problem we'd like to tackle is to extend the end-to-end tests with a scenario for chain & client upgrades. Currently we only have some manual testing guide for this scenario.

Proposal

investigate and find which event does a chain generate upon upgrading
- depends on https://github.com/cosmos/ibc-go/issues/270
add mechanism to handle the event, so that Hermes automatically triggers client upgrade
all clients (which target the upgraded chain) hosted on one of the chains in the config file should be upgraded

Acceptance Criteria

[ ] support for reactive client upgrade
[ ] extension to E2E test suite with appropriate test scenario for chain & client upgrade

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate milestone (priority) applied
[ ] Appropriate contributors tagged
[ ] Contributor assigned/self-assigned

informalsystems / hermes