examples: cw-orchestrator using starship

[Anmol1696]

Overview

Currently AbstractSDK/cw-orchestrator is using a fork of interchain tests with a mix of golang and rust coupled with docker. We create an example in which we are able to perform the various e2e tests

Proposal

Inorder to achive this we have to do the following

On Starship: Rust Client

Create a rust client similar to the go client: This client will be responsible for initial connection with the starship infra running from a config file.

The client will do the following:

• Have information about all the chains running, relayers etc
• Initial keys/mnemonics with the balances
• Handy util functions to: send tokens, ibc tokens, contract deployment (?), contract store (?) etc

On cw-orchestrator: Starship directory

First we create a starship dir in root, with the

• config file (specifying the infra we want to setup)
• Makefile with handy commands to spin up starship infra, and to connect to it
• Test cases, which will import the rust client created above, and write idempotent test cases

Bonus:

• Add CI/CD using starship-action to run the whole testing setup and test cases (maybe in a k8s cluster)

Workflow

Once we have the components the tests would look something like:

# Spin up the infra
make start

# Run the tests, should be able to run multiple times
cargo test

# Stop everything
make stop

The setup is such that there is some initial setup time, so it would make sense to write test cases itself such that they can be run multiple times against the same infra setup.

Note: In CI/CD we spinup the system from scratch.

[CyberHoward]

Awesome man! For our use case we'd be happy with a minimalist client, i.e. exposing only the starship specific functionality. We will almost certainly parse the infrastructure information into Daemon objects that will back the dev's contracts and those already have all the nice SDK util functions that you can think of.

So the way I see it working on the cw-orch side is that we add a StarshipClient that can be constructed easily and that is able to resolve the starship infrastructure details (like the InterchainInfrastructure object here. The dev can then get access to the specific chain by retrieving the Daemon from this infrastructure object like so. This way we re-use all the functionality that we already have for interacting with blockchain nodes. Your client should be able to replace or back the InterchainInfrastructure struct for this functionality.

For the IBC part of the integration it would be really nice to have a function that creates an IBC channel between two ports, that way we can easily expose that to devs.

So as a dev I would:

1. Create a starship config file
2. Write a test that instantiates a Starship object that I can get the Daemon(node) objects from.
3. Spin up the environment with make start
4. Run the test(s)
5. Stop the env when everything is done.

What do you think?

Sidenote: In Rust there's a way to run code right before and right after integration test execution, we use this feature already to test cw-orch's daemon code by spinning up a local node before integration testing, with a little macro (https://github.com/AbstractSDK/cw-orchestrator/blob/2a7bfc312393b2db0de32cdd6fca78f4504c89f2/cw-orch/tests/common/mod.rs#L174) and then run our tests against the infra and shut it down after the tests. The nice thing about this is that it is extremely reproducible and the only requirement for the dev to run this test is having Docker installed. Something similar but with starship as the backend would be super dope but probably a lot harder to do because of the software requirements.

[Anmol1696]

This is really helpful for the development. Makes sense, then the client in rust is even lighter, just ability to understand what is going on. Yes spinning up something as part of setup of the test itself would be really nice.

Q1) I think we just need to think about making the setup idempotent too? What are your thoughts on, test setup will spin up the infra only if it does not exists or there is a flag to force restart the infra, that way we can get the best of both worlds.

I will try and get this working with as little modifications to your system as possible, that would be the best bit.

For the IBC stuff, I will add the functionality that you mentioned as well, currently the transfer ports are the only ones opened between 2 chains, and this is in the config file. For examples:

chains:
- name: osmosis-1
  type: osmosis
  numValidators: 2
- name: juno-1
  type: juno
  numValidators: 2

relayers:
- name: osmo-juno
  type: hermes
  replicas: 1
  chains:
  - osmosis-1
  - juno-1

With this config, when the relayer spins up between the chains, the first conneciton is created during the spinup time itself. The information about which connection/port/channel is between which chains is in the registry service (similar to the actual chain-registry repo). This information will be available to the devs with the StarshipClient.

Q2) Currently are there any other IBC ports you are interested in opening (apart from Transfer port)? I will have to look into how to provide a better way to create the channels on the fly as well?

[CyberHoward]

I'm liking these vibes. 1) I like the idea very much, only thing is that account balances are set at genesis so token transfers can theoretically run out(?). If there is an bank(?) mint endpoint that you could point me to than I could solve that. For doing startup/reset operations I'm assuming that would still require running make commands? If we can do it programmatically that would be super nice. If not then I think your earlier suggestion of having a remote server host the infra would be a better option.

Having the transfer channels already set up is great so thanks for that! The CW channels can just re-use the connections so if those are queryable some way than we should be golden.

2) Yes, on-the-fly channel creation is a hard requirement as a contract's IBC port is essentially it's address, which is only effectively known when the contract is instantiated, hence we have this create_channel function on our Hermes object. So we need a way to get the channel information and some way of executing this command on the hermes container.

Also I'm assuming the hermes container is set up to relay every packet committed to the connection, not just the transfer channel?

[Anmol1696]

The CW channels can just re-use the connections so if those are queryable some way than we should be golden.

Yup they are. I will show the way how as well.

Also I'm assuming the hermes container is set up to relay every packet committed to the connection, not just the transfer channel?

Yes again, it is open. I will have a look into how to expose the channel creation better on Hermes. I dont like to run a shell script in the docker container, i rather have a side car with the container on which you can send a request to create the channels and what not. Let me add that functionality too properly as well.

I have a design in which i run exposer (which is basically a hacky) side car to do all the dirty work on the container. Will make it such that you can create channels as well there. (worst case scenario would be give it the command to run and it runs the command on the hermes). Currently not alot of features of the relayer i expose, but this is a really good usecase to get that working as well.

Thanks for the feedback, this is a good direction to take the product and build out Starship in a generic way.

[Anmol1696]

account balances are set at genesis so token transfers can theoretically run out(?). If there is an bank(?) mint endpoint that you could point me to than I could solve that

I prefer the approach of having facuets here instead. We can have a mint module exposed, but that would mean forking the chain which i would prefer not to. We can initialize insanely high number of tokens, then use the facuets to tansfer the tokens there.

The test setup is not meant to be run for too long, so if the tokens do run out, just delete and restart the infra is the best way forward.

We can look into this once we start facing this issue, i am sure we can figure out a good solution for this too, if this becomes a problem for sure.

[Anmol1696]

@CyberHoward Could you link here to your work? I could use it for getting the initial rest client in place.

[CyberHoward]

Here is the initial client I wrote up: https://github.com/AbstractSDK/cw-orchestrator/tree/feature%2Fstarship/packages%2Fcw-orch-starship