Unblock TMKMS Integration for validator keys

[anilmurty]

While Scott has confirmed (and documented) that Akash validator nodes deployed on Akash, can work with TMKMS (https://docs.akash.network/other-resources/experimental/omnibus/akash-validator-with-tmkms) -- we can't make this generally available until we figure out a way to deal with a proxy/ vpn/ traefik at the ingress controller.

This is a priority because:

1. Validators want to be able to store keys securely (using TMKMS or similar)
2. Omnibus repos will not get traction until it can support secure key storage (through TMKMS) because validators will not use a TMKMS instance that is "open to the world"
3. Console depends on Omnibus for validator/ chain node templates, so Console cannot release (it can release but no validators will use) templates that do not support a "secure TMKMS" - that has an ingress controller capable of filtering traffic based on source IP (restricting to just the validator's node)

[boz]

refs ovrclk/engineering#307

[andy108369]

@tombeynon created https://github.com/ovrclk/stunnel-proxy repo where one can expose as many services as needed. Whatever runs in between the stunnel client & server will get mTLS authenticated+authorized & encrypted.

I've tried it and made a demo running an app behind the stunnel-server in Akash Network here https://asciinema.org/a/519302 (both SDL files are in the description there).

Upd: Tom is going to test this against a validator this week.

[andy108369]

Good news from Tom on the stunnel-proxy (mTLS) protecting his validator <-----> TMKMS (& RPC) communication:

• His Kujira testnet validator is signing with a node/proxy on Akash, and KMS/proxy on his servers;
• It works great. It's a fast blockchain (~2s) but not missing any more than usual;
• Proxying RPC too, he is using that with his monitoring tools. All works perfectly;
• He will start writing up a bit of guide in Omnibus _examples;
• It still needs making the stunnel-proxy docker image publicly available (Adam is on it);