search

ubiquity / ubiquity-dollar

Ubiquity Dollar (UUSD) smart contracts and user interface.
https://uad.ubq.fi
Apache License 2.0
34 stars 90 forks source link

GitHub Actions private testnets #209

Closed 0x4007 closed 1 year ago

0x4007 commented 2 years ago

I propose that the next/final(?) step to make Sergey's vision work is to have GitHub actions compile [...]

Originally posted by @pavlovcik in https://github.com/ubiquity/ubiquity-dollar-development/issues/193#issuecomment-1182551791

So GitHub Actions runner can run for 6 hours with something crazy like 16GB memory, meaning, we could probably have the beta testing UI do an AJAX request to tell GitHub to build our contracts and host an Anvil instance, and then return the URL + Port of that runner to use that instance of the Anvil blockchain!

Broken down into subtasks as per @sergfeldman suggestion:

nshmadhani commented 2 years ago

I would like to take on this bounty if it is still available!

0x4007 commented 2 years ago

It's still available! Do you have any more questions about the bounty?

nshmadhani commented 2 years ago

Yes I do, I wanted to know if there any specification of what needs to be implemented. I have worked on this a little bit and im stuck on getting hardhat instance url for the frontend.

  1. is there a deployment script that needs to be ran along with the node?
  2. how will the user select trigger the action runner?
0x4007 commented 2 years ago
  • is there a deployment script that needs to be ran along with the node?

This is awkward timing as we are in the middle of converting the project from Hardhat to Foundry. @0xcodercrane seems to have removed the old shell scripts used for quickly starting the Hardhat node and UI in parallel. For example, this one I remember being pretty useful to automatically run the faucet after the Hardhat node is operational. You could check that directory for more tools we used to duct tape the dev process together.

Anyways, I think that it might be best to wait for the dust to settle with the Hardhat to Foundry conversion and then we can consult with @0xcodercrane on the best way to start Anvil, and then packaging it to be able to be invoked from an API call (from our UI) and start an instance using a GitHub Actions Runner.

  • how will the user select trigger the action runner?

I haven't deeply done research on implementation details but I assume that GitHub has an API that we can make a request to, to start a GitHub Actions Runner instance. Once the instance is running, I remember reading that it has very generous free limits, including being able to run a script for like six hours. This script can run Anvil and expose the RPC endpoint over the network which we can connect to using our MetaMasks from the frontend. The UI must of course expose the unique GitHub Actions Runner URL once the instance is started to allow the beta tester to connect to it using MetaMask.

Hope that clears things up!

@0xcodercrane any estimate on timeline for when the dust will settle on the conversion?

0xcodercrane commented 2 years ago

My suggestion to setup beta testing environment is basically as the following. First of all, I don't think there is an API to access hosted runners every time github action runs.

The estimation on timeline for foundry conversion is sort of the end of the month. but @nshmadhani can start working on it if you want to take over. The full list of subtasks which should be done on this issue are

@sergfeldman @pavlovcik please feel free to update described above if there is a missing/wrong part.

0x4007 commented 2 years ago

First of all, I don't think there is an API to access hosted runners every time github action runs.

  1. Create a workflow dispatch event

You can use this endpoint to manually trigger a GitHub Actions workflow run.

  1. Get a workflow

Gets a specific workflow.

0x4007 commented 2 years ago

Objective

Sometimes I wonder if the testing infrastructure we envisioned is too robust and complex for most needs, but as long as we can make it convenient for non developers to jump in, test, and offer feedback; then we can consider this business objective completed. If we make it attached to GitHub Actions, in theory the testnet deployed contracts should be automatically in sync with our latest code. This makes future maintenance virtually non-existent for our perpetually ongoing beta testing program.

Perhaps @sergfeldman has some input here.

Costs

[ ] I guess the way to run the hardhat chain is that we host the hardhat chain somewhere(vps, heroku, cloudflare, etc).

The point of GitHub Actions is that it's free and powerful. Imagine if we have 100+ beta testers (one could hope) with separate Anvil instances. There's no way a tiny VPS could handle it. Our build process alone uses 2-3 GB of RAM last I tested FYI. Otherwise we pay for a powerful VPS and burn a ton of unnecessary money for when people stop beta testing.

Benefits

The original vision was to allow beta testers to "rewind" the chain state by reinitializing/redeploying the Anvil instance. Obviously if we have more than one tester connected to the same instance this would be annoying.

Or even better, deploying a separate Anvil instance and being able to switch between chain states/instances easily right from the UI! That would make user testing really dope imo.

If we really can't figure this out, then we mind as well deploy on a test network like Ropsten, perhaps?

nshmadhani commented 2 years ago

Using github actions runner might be too complicated for this task, since have no way of connecting to the anvil instance once the fork has been launched on a private testnet. As @0xcodercrane said we also need to include a subdomain for each of these instance once they are launched this process itself could be time consuming.

I have tried running this action in test with hardhat and once the instance is launched github actions steps pauses and wait s for the instance to shutdown, any subsequent steps to deploy contracts do not execute. any ideas on how i can do them parallely?

0x4007 commented 2 years ago

since have no way of connecting to the anvil instance once the fork has been launched on a private testnet

Just host it on 0.0.0.0:* I know that GitHub Codespaces exposes these through tunnels that can be connected from remotely. I assume that actions might have some similar capability.

Another workaround might be to use another serverless solution but I'm pretty confident that GitHub can do it all.

need to include a subdomain for each of these instance

I already have dynamic subdomain support enabled. We just need a reverse proxy to point at each of these instances when they are instantiated.

codeawayaks commented 2 years ago

You can assign this to me.

sergfeldman commented 2 years ago

You can assign this to me.

@codeawayaks You commented on 20+ GitHub issues that you want to take it. One developer should solve one bounty at a time to minimize blocking other developers to effectively move forward. Which 1 bounty do you want to take?

0x4007 commented 1 year ago

One possibility:

  1. Start Anvil
  2. Use ngrok to tunnel the port and expose to a public IP

https://twitter.com/fiveoutofnine/status/1583501045225107461?s=46&t=qb_SISQVHNsKvXCWKhqhwA

nshmadhani commented 1 year ago

I thought of this but the issue was that for github actions. You cannot run commands in parallel. Once anvil is started, starting a tunnel is difficult. you would need to somehow run this in parallel. Also, sending data back from the github actions is not well documented.

Just posting this incase, if someone tries it out.

0x4007 commented 1 year ago

You cannot run commands in parallel. Once anvil is started, starting a tunnel is difficult. you would need to somehow run this in parallel.

Why not just append & to daemonize the process?

0x4007 commented 1 year ago

Workflows can be triggered by apps outside of GitHub

0x4007 commented 1 year ago

I spent the last couple weeks looking more closely into how GitHub Actions work, and I believe that we can pretty easily make a GitHub App (and host on the GitHub Marketplace) that will be able to run Anvil instances, and expose an RPC endpoint.

This would make for a useful product outside of just our internal development efforts.

hashedMae commented 1 year ago

Do we know if this would create a consistent RPC endpoint on each spin up or would that need to be retrieved by the front end for testers to then add to their wallet for each cycle?

If this is for a staging environment I don't understand why we don't use existing testnets?

0x4007 commented 1 year ago

Do we know if this would create a consistent RPC endpoint on each spin up or would that need to be retrieved by the front end for testers to then add to their wallet for each cycle?

I envision that all live instances will have unique RPC endpoints that have a lifespan of 6 hours. All UIs should be able to see every active RPC endpoint for beta testing. The user should be able to click to add the RPC endpoint to their MetaMask to connect to that blockchain.

If this is for a staging environment I don't understand why we don't use existing testnets?

Forking and being able to rapidly test scenarios in parallel! For example, we had to pay without testing https://github.com/ubiquity/ubiquity-dollar/pull/279 because we were unable to manipulate the TWAP easily (the swap interface was down while I personally was trying to test.) It would be quite a luxury to be able to get the support of another dev to set up and manipulate this environment/chain-state to be able to complete the testing of this particular bounty; meanwhile it would be great to be able to do the same for other bounties. With Anvil we would theoretically be able to spin up 20 test networks simultaneously testing 20 different bounties at the same time which require different chain states.

I also elaborated a bit earlier in this conversation if you scroll up. I think it could be a really useful development tool for our and other crypto projects.

acaldas commented 1 year ago

I'd like to work on this. I did a test and was able to send JSON RPC requests to an Anvil process running on a Github Action.

Another advantage of using this over a testnet is that we can use methods like anvil_impersonateAccount and evm_increaseTime to alter the state of the chain to test specific scenarios. With anvil_dumpState/anvil_loadState we should also be able to have some pre-defined test scenarios, we could even have an option in the UI to deploy them.

Here is how I think this feature could work:

  1. We have a trigger on Pull Requests, like adding a deploy anvil label or a comment in the PR, that triggers the Github Action to run the Anvil fork with Ngrok.
  2. The action triggers a Github Deployment for the PR, this makes the deployments easier to manage. (I think we can even set a limit of parallel deployments).
  3. The deployment is listed on the PR as Netlify does currently. A user can then send RPC requests to alter the state to his needs and use it through Metamask.

The developer experience can be increased significantly in the UI. The available Anvil instances can be listed by using the Github Deployments API. We could also add a way to send pre-defined JSON RPC requests from the UI for funding an account and other common requests.

0x4007 commented 1 year ago

I really appreciate your ideas and research. I'm also glad that you see the potential of this infrastructure. Happy to assign this to you and to contribute to the research with you!

I think we might need to break this up into several smaller tasks, such as:

(Moved to first comment so that I can easily create subtasks, a quirk with the GitHub UI I guess)

sergfeldman commented 1 year ago

I really appreciate your ideas and research. I'm also glad that you see the potential of this infrastructure. Happy to assign this to you and to contribute to the research with you!

I think we might need to break this up into several smaller tasks, such as:

  1. Setting up the CI
  2. Setting up the dev UI
  3. Anvil dump/load state
  4. Listing all available Anvil instances
  5. Change balance (faucet)
  6. Impersonate account
  7. Custom json rpc request from UI

@acaldas This issue is quite big and must be split to smaller tasks with specific deliverables.

Do you agree with this list ^ of smaller tasks? If yes, then please provide estimate of a reasonable price and time for each task. If not, then please suggest our list of smaller tasks.

0x4007 commented 1 year ago

Moved the tasklist to the first comment so that I can quickly create subtasks.

acaldas commented 1 year ago

Sorry for the lack of updates, I was out this week.

Yeah, it would be helpful to split it into subtasks. I think it would make sense to start with an MVP solution going through the whole flow: PR -> Trigger GitHub Action for Anvil with Ngrok -> Create Deployment in Github (and close it when Anvil stops running) -> Notify PR -> List Deployments in UI

Then, once that solution is accepted, it would be more about nice-to-have features like doing the JSON-RPC requests through the UI and making it easier to manipulate the chain state for different test scenarios. Some of these improvements will probably come up once people start using this tool.

sergfeldman commented 1 year ago

@acaldas Start with an MVP sounds good. Please, analyze the list of tasks compiled by @pavlovcik (the first comment) and propose tasks for an MVP

rndquu commented 1 year ago

Testnets

User stories

Approaches (pros and cons)

  1. VPS
  2. tenderly
  3. javascript EVM instance (https://github.com/ethereumjs/ethereumjs-monorepo/tree/master/packages/vm)
  4. github actions + anvil + tunneling (ngrok or cloudflared)
free does not consume resources when nobody is using well documented near productuon environment
VPS - - + +
tenderly - + + +
js EVM + + - -
github actions + + + +

How it should work

  1. User wants to test a specific project feature or a newly created PR
  2. For the desired branch user runs a "deploy test blockchain" workflow which:
    • starts anvil in the github action runner
    • deploys contracts(in their initial state) into the newly started anvil instance
    • opens local port (where anvil is listening) to the world (cloudflared, ngrok)
    • prints RPC URL in the workflow logs
  3. When user finishes testing the PR he stops the workflow along with the anvil instance
  4. There should also be a timeout of inactivity which stops the workflow in case user forgot to stop the workflow himself

So the end user flow is the following:

  1. User wants to test the development branch or PR is created
  2. User starts the "deploy test blockchain" workflow on the desired branch
  3. RPC URL is posted in the workflow logs
  4. User connects to RPC URL
  5. User sends custom RPC requests (https://book.getfoundry.sh/reference/anvil/#custom-methods) to modify blockchain state in order to test the feature
  6. Users stops the "deploy test blockchain" workflow

Tasks to create

We already have a PR for starting anvil and tunneling (https://github.com/ubiquity/ubiquity-dollar/pull/442/). I think it can be merged.

1. Deploy contracts to anvil instance when "deploy test blockchain" workflow starts

Depends on: https://github.com/ubiquity/ubiquity-dollar/pull/417

So when https://github.com/ubiquity/ubiquity-dollar/pull/417 is ready we should deploy contracts in their initial state when "deploy test blockchain" workflow starts. The "deploy test blockchain" workflow can be started only manually.

2. Print RPC URL when "deploy test blockchain" workflow starts

When user runs the "deploy test blockchain" workflow it should print the deployed RPC URL in the workflow logs.

3. Add "deploy test blockchain" workflow timeout

Implement a timeout for the "deploy test blockchain" workflow so that when nobody uses the deployed node for some time the instance should be automatically closed.

4. Create UI for custom RPC methods

When user wants to test a PR he should be able to update blockchain's state for testing purposes.

Anvil provides custom RPC methods for better testing: https://book.getfoundry.sh/reference/anvil/#custom-methods

What should be done:

  1. Add "Custom RPC methods" button to the bottom of the https://uad.ubq.fi/ (button should be hidden for production builds)
  2. On "Custom RPC methods" click popup should be opened where user can send txs for the following RPC methods:
    • anvil_impersonateAccount
    • anvil_stopImpersonatingAccount
    • anvil_setBalance
    • anvil_setStorageAt
    • anvil_dumpState
    • anvil_loadState
    • evm_increaseTime
    • eth_sendTransaction
0x4007 commented 1 year ago

@rndquu I want to clarify that my vision of this feature is primarily to allow beta testers to test specific features (any, simultaneously) with custom chain states from the UI. In this vision it is not necessarily dependent on a specific pull request.

For example, I want to test a Ubiquity Credit related feature that is dependent on the current market price of the Ubiquity Dollar. The beta tester can go to the UI, open the "beta testing tools" view, and press a button to launch a new chain.

Once that chain is created, the user can then manipulate the chain state to allow them to test the specific (price-dependent) feature. In this case, they would impersonate a Ubiquity Dollar whale, and perform a large market buy (or sell) to manipulate the price. I think in the early days it would have to be from their command line to fire off custom RPC requests, and eventually maybe we can drop in a UI component for this.

When the user is done testing that feature, and lets say that the user wants to test the opposite state of the UI simultaneously (let's say the first example was for above-peg, and the second example for below-peg UI state.) the user can start another chain, do the market sell and then test the UI.

In this vision, the beta tester can quickly switch between both chain states to test above and below peg UI at the same time. This should make testing a pleasant experience. The beta tester can also easily ask the other non-technical people on the team to connect to both RPC endpoints and test as well, for the next six hours, all for free!

rndquu commented 1 year ago

@rndquu I want to clarify that my vision of this feature is primarily to allow beta testers to test specific features (any, simultaneously) with custom chain states from the UI. In this vision it is not necessarily dependent on a specific pull request.

For example, I want to test a Ubiquity Credit related feature that is dependent on the current market price of the Ubiquity Dollar. The beta tester can go to the UI, open the "beta testing tools" view, and press a button to launch a new chain.

Once that chain is created, the user can then manipulate the chain state to allow them to test the specific (price-dependent) feature. In this case, they would impersonate a Ubiquity Dollar whale, and perform a large market buy (or sell) to manipulate the price. I think in the early days it would have to be from their command line to fire off custom RPC requests, and eventually maybe we can drop in a UI component for this.

When the user is done testing that feature, and lets say that the user wants to test the opposite state of the UI simultaneously (let's say the first example was for above-peg, and the second example for below-peg UI state.) the user can start another chain, do the market sell and then test the UI.

In this vision, the beta tester can quickly switch between both chain states to test above and below peg UI at the same time. This should make testing a pleasant experience. The beta tester can also easily ask the other non-technical people on the team to connect to both RPC endpoints and test as well, for the next six hours, all for free!

I've updated the description https://github.com/ubiquity/ubiquity-dollar/issues/209#issuecomment-1383632754

Now users can start the "deploy test blockchain" workflow from any branch. So when user starts the "deploy test blockchain" workflow from the development branch it is enough for him to test specific project features on custom chain states. Testing PRs is also supported as it is simply running the "deploy test blockchain" workflow from the PR branch.

The proposed solution is not very user friendly because anvil instances are started from github actions not from the UI. In the next iteration we can think of running anvil instances from the project UI. But I doubt that starting anvil instances is possible without a backend serivce because in order to use github API you need an API token but you can not save it on the client side. So you need to call some backend service which stores github API token and this backend service would start workflows.

0x4007 commented 1 year ago

We already have a PR for starting anvil and tunneling (https://github.com/ubiquity/ubiquity-dollar/pull/442). I think it can be merged.

Merged!

But I doubt that starting anvil instances is possible without a backend serivce because in order to use github API you need an API token but you can not save it on the client side. So you need to call some backend service which stores github API token and this backend service would start workflows.

We could use tightly scoped permissions and make the API key basically public/disposable. Further, we can make a dedicated repository just to host the GitHub Actions related to the testnets. With careful consideration to permissions and making a dedicated repository, perhaps a possible attack surface is entirely mitigated from anything sensitive?

rndquu commented 1 year ago

@pavlovcik

perhaps a possible attack surface is entirely mitigated from anything sensitive?

this needs further research

anyway, I think we need more reviews from the team for the proposed tasks here https://github.com/ubiquity/ubiquity-dollar/issues/209#issuecomment-1383632754

@ubiquity/development , @acaldas guys what do you think?

hashedMae commented 1 year ago

For example, I want to test a Ubiquity Credit related feature that is dependent on the current market price of the Ubiquity Dollar. The beta tester can go to the UI, open the "beta testing tools" view, and press a button to launch a new chain.

Once that chain is created, the user can then manipulate the chain state to allow them to test the specific (price-dependent) feature. In this case, they would impersonate a Ubiquity Dollar whale, and perform a large market buy (or sell) to manipulate the price. I think in the early days it would have to be from their command line to fire off custom RPC requests, and eventually maybe we can drop in a UI component for this.

To impersonate an account you need to pass a command to Anvil through a CLI. This is effortlessly done on local. Doing it through GitHub actions, while cool, seems needlessly complicated.

rndquu commented 1 year ago

For example, I want to test a Ubiquity Credit related feature that is dependent on the current market price of the Ubiquity Dollar. The beta tester can go to the UI, open the "beta testing tools" view, and press a button to launch a new chain. Once that chain is created, the user can then manipulate the chain state to allow them to test the specific (price-dependent) feature. In this case, they would impersonate a Ubiquity Dollar whale, and perform a large market buy (or sell) to manipulate the price. I think in the early days it would have to be from their command line to fire off custom RPC requests, and eventually maybe we can drop in a UI component for this.

To impersonate an account you need to pass a command to Anvil through a CLI. This is effortlessly done on local. Doing it through GitHub actions, while cool, seems needlessly complicated.

we're talking about impersonating accounts from UI side, in particular from https://uad.ubq.fi/

it just hit me that if we want to impersonate account from the UI side we should also implement eth_sendTransaction RPC method (task 4 from in https://github.com/ubiquity/ubiquity-dollar/issues/209#issuecomment-1383632754) because if we want to impersonate an account for some user whose private key is unknown the flow should be this one:

  1. from the UI side call custom RPC method anvil_impersonateAccount
  2. from the UI side call RPC method eth_sendTransaction and set impersonated address in the from field
  3. from the UI side call custom RPC method anvil_stopImpersonatingAccount

added eth_sendTransaction to task 4 in https://github.com/ubiquity/ubiquity-dollar/issues/209#issuecomment-1383632754

acaldas commented 1 year ago

To hide the GitHub API key from the client side we could use a Clouflare Worker as a proxy. The client sends a request to the Worker, which could filter the requests to only allow from our deployment domains. The Worker would then send the request to the GitHub API. The Free plan allows 100,000 requests/day.

I think that a useful feature would be to have a folder with test scenarios. For instance, when a dev opens a PR, he could add an Anvil script that sets up the necessary state to test the changes of the PR. Then on the UI someone can spin up an Anvil instance and select the script to be run.

rndquu commented 1 year ago

@acaldas

we could use a Clouflare Worker as a proxy

Yes, this is a server side approach. I mean that Github API key would be hidden in Clouflare Worker. But I'm afraid that adding such a proxy backend would add an additional layer of work for maintaining this server proxy. So I think we can leave this proxy backend for the next "private testnets" feature iteration.

when a dev opens a PR, he could add an Anvil script that sets up the necessary state to test the changes of the PR

Yes. The dev who opened the PR could also provide a whole chain state (from anvil_dumpState) which could be uploaded via anvil_loadState

0x4007 commented 1 year ago
  1. Deploy contracts to anvil instance when "deploy test blockchain" workflow starts

We shouldn't automatically do this because we can only have so many active actions running and we push a ton of commits daily; each would be blocking one action slot for six hours. Instead, these should be manually invoked from the beta testing UI as that will happen much more rarely.

@rndquu

rndquu commented 1 year ago
  1. Deploy contracts to anvil instance when "deploy test blockchain" workflow starts

We shouldn't automatically do this because we can only have so many active actions running and we push a ton of commits daily; each would be blocking one action slot for six hours. Instead, these should be manually invoked from the beta testing UI as that will happen much more rarely.

@rndquu

The "deploy test blockchain" workflow is supposed to be run only manually. For the 1st "private testnets" feature iteration user will have to call a workflow manually from github actions UI. In the next "private testnets" feature iteration we can consider calling the workflow from beta testing UI(i.e. from https://uad.ubq.fi/)

rndquu commented 1 year ago

I've removed "time" and "priority" labels because this is a meta issue with subtasks which should not be self assigned directly

0x4007 commented 1 year ago

@rndquu looks like this is completed so we can mark as complete right?

ubiquibot[bot] commented 1 year ago

Permit generation skipped since the issue is identified as parent issue.

If you've enjoyed your experience in the DevPool, we'd appreciate your support. Follow Ubiquity on GitHub and star this repo. Your endorsement means the world to us and helps us grow!
We are excited to announce that the DevPool and UbiquiBot are now available to partners! Our ideal collaborators are globally distributed crypto-native organizations, who actively work on open source on GitHub, and excel in research & development. If you can introduce us to the repository maintainers in these types of companies, we have a special bonus in store for you!

rndquu commented 1 year ago

@rndquu looks like this is completed so we can mark as complete right?

Yes, we have everything ready to use anvil from github runners