Improve Testing for Geth

[abdulrabbani00]

Current Testing

Currently, there are two sorts of tests that developers can run for vulcanize-geth.

1. Unit Tests
2. Manual Integration Tests

Unit Tests

The unit tests are written directly in the vulcanize-geth code base. They follow the same standard as ethereum-geth unit tests.

Manual Integration Tests

If we want to test the behavior of a running vulcanize-geth node, on a private network, we need to do the following:

1. Utilize dapptools to run geth and create a private network.
2. Manually study behaviors of our running geth node.

Although manual integration tests are helpful in many contexts, primarily to study code change behavior at the start of development, they don’t provide full code coverage. Furthermore, the current process for standing up a private network and connecting it to Geth spans two repositories and takes and takes a long time to build.

Improvements

@Ian Norden and I propose the following two solutions:

1. Adding Hive (or another framework) for programmatic integration tests for geth.
2. Utilizing Foundry to build a private network, perform manual integration tests, and test smart contacts (down the road).

Ideology

• The sooner a testing framework is integrated into a codebase or process, the sooner it can be leveraged. This means others can efficiently utilize this framework to test their new changes.
• If we push off adding testing frameworks for a later date, new features in the intermediary will not be able to leverage those testing frameworks.
• In most cases, it is very unlikely for developers to go back to past code changes and add new tests when a new framework is developed.
• With all this in mind, if we add programmatic integration tests at this stage, we can have a higher chance of writing these tests for upcoming features.

Programmatic Integration Tests

There is no way to write proper programmatic integration tests against the geth codebase. There are various issues open now that could leverage programmatic integration testing.

Proposal

1. Look into Hive and see if it will cover our needs for writing programmatic integration tests.
   1. Look into other integration testing frameworks for geth.
2. Once a framework has been decided, include why it was chosen in this document. This reasoning will provide insight to future developers.
3. Fork the testing framework.
4. Create a generic pipeline/process for being able to test vulcanize-geth.
   1. This process will need to include a connection to the ipld-eth-db.
5. Add a simple “dummy” test to ensure that the testing of vulcanize-geth works.

Immediate Use Cases

• https://github.com/vulcanize/go-ethereum/issues/129 - This issue will require integration testing to ensure we handle all use cases of a dropped DB connection.

Replace Dapptools with Foundry for Manual Integration Testing on a Private Network

Currently, the pain point with dapptools is:

1. It is in a completely separate repository as vulcanize-geth.
2. It is very slow to build. It contains a lot of “fluff,” which doesn’t provide a great deal of value.

The benefits to Foundry are:

1. It is more robust and faster to “spin up.”
2. It allows a built-in capability to test smart contacts in solidity.
3. We don’t need to maintain a fork to use it.

Proposal

1. Create a folder in vulcanize-geth for foundry.
2. Include a process that will create a private network for manual integration testing. The process will more than likely do the following:
   1. Compile the local version of geth.
   2. Creates a container with the ipld-eth-db
   3. Create a container that contains Foundry. In this container, we will mimic the actions that dapptools performs to create a private network:
      1. Start geth
      2. Deploy contracts
      3. Send transactions.

Immediate Use Cases

• https://github.com/vulcanize/go-ethereum/issues/181 - Including Foundry in the codebase will allow us to quickly compile geth, connect it to an ipld-eth-db, and run it in GitHub actions.

Estimated Time

The following captures the estimated time and effort required to make both changes.

Foundry

• I believe the effort here might be fairly light.
• Running Foundry in a container and writing a docker-compose file seems pretty straightforward.
  • Hopefully, there is already a public image for Foundry.
• The only place I can see myself having issues is attempting to mimic creating the private network.
  • Dapptools does it using a bash script. I believe a lot of it will be “copy-pasting” what dapptools is doing and making improvements.
    • I can see issues occur during the “copy-pasting” due to a lack of understanding of how the private network is created. But I believe debugging those issues can provide me with insight into private network creation.
  • I might also take some time to browse around and see if there is a more robust way of creating and starting a testnet. However, it might be best to mimic what we have in dapptools for the time being.
• Hopefully, this process will take me less than ~4 hours. I think that is very feasible, and it might be significantly less if I do not run into any issues with creating the private network.

Hive or Hive-Like Solution

• I would want to take some time to see if there are other solutions out there that might better fit our needs.
• Forking Hive and running the out-of-box integration tests should be pretty straightforward.
• Add a process for testing a local version of vulcanize-geth
  • Here is where things can get tricky.
  • I would want to add the ability to run vulcanize-geth and connect it to an ipld-eth-db
    • Then trigger Hive to run tests against vulcanize-geth.
• Once the process for setting up Hive to test vulcanize-geth has been implemented, I would want to do the following:
  • Create a simple CI/CD pipeline within the vulcanize-geth repository that runs the tests in Hive.
  • Add a straightforward test case for statediffing to ensure that everything is running as expected.
• The estimated time for this is ~4-8 hours. Hopefully, it will be on the lower end, but I can see myself having issues with integrating vulcanize-geth into Hive.

[i-norden]

LGTM!

For the Foundry proposal 2.ii. "Creates a container with the ipld-eth-server" not sure I follow why the container in this rpeo would need to include ipld-eth-server.

[abdulrabbani00]

Good catch, I meant the ipd-eth-db not server.

[abdulrabbani00]

Status Updates

• Forge has been integrated into the vulcanize/go-ethereum repository. But it might be moved into a repository of its own.
  • Developers can utilize it for local testing.
• Hive has been integrated into our organization.
  • Developers can utilize it for local testing.
  • Hive uses the aforementioned foundry changes to run Geth and test against it.

Going Forward

Currently, this framework works for local testing, but it's not great and hasn't been standardized. The next steps are:

• Get feedback from other developers on improvements.
• Write an integration test for statediff using Hive.

Once we have gotten appropriate feedback and used both frameworks for real-world testing, we can do the following:

• Build a CI/CD pipeline(s), depending on developer needs.
• Maintain docker images, depending on developer needs.
• Push dapptools out of existing pipelines.

[i-norden]

Reopen more specific task issues