E2E testing with docker compose

worldcoin / signup-sequencer

signup-sequencer repository

MIT License

67 stars 33 forks source link

E2E testing with docker compose #727

Closed piohei closed 2 months ago

piohei commented 3 months ago

Motivation

This E2E testing will be used mostly for HA solution testing. We need to run multiple instances of sequencer to ensure our prod env will work properly.

Solution

E2E tests are basicly running docker compose under the hood and execute some requests to follow different kind of scenarios. This docker compose can be also used for local development.

PR Checklist

[ ] Added Tests
[ ] Added Documentation

piohei commented 2 months ago

I've added option for parallel run. There are few things left:

Is it safe to commit into this repo keys for insertion prover and deletion prover?
I'm not sure if I've done right things with committing docker compose with anvil args where I use url for fork that includes API key for alchemy.
When do we want to run these tests? They will be the slowest ones.

Dzejkop commented 2 months ago

I've added option for parallel run. There are few things left:

Is it safe to commit into this repo keys for insertion prover and deletion prover?

I'm not sure if I've done right things with committing docker compose with anvil args where I use url for fork that includes API key for alchemy.

When do we want to run these tests? They will be the slowest ones.

We can include keys no problem. The API keys is a bit problematic though. Maybe we can pass it through GH secrets?

Regarding when to run these - how about we set a cronjob and run it a couple times a day?

Dzejkop commented 2 months ago

how about we set a cronjob and run it a couple times a day? alternatively everytime we push a tag?

piohei commented 2 months ago

how about we set a cronjob and run it a couple times a day? alternatively everytime we push a tag?

Tag sounds good to me and makes sense. We will treat it as "last checkpoint" before going on with deploy. :)

piohei commented 2 months ago

I've added option for parallel run. There are few things left:

Is it safe to commit into this repo keys for insertion prover and deletion prover?

I'm not sure if I've done right things with committing docker compose with anvil args where I use url for fork that includes API key for alchemy.

When do we want to run these tests? They will be the slowest ones.

We can include keys no problem. The API keys is a bit problematic though. Maybe we can pass it through GH secrets?

Regarding when to run these - how about we set a cronjob and run it a couple times a day?

I will set this API key as env. Then we can use GH secrets for CI. :)

piohei commented 2 months ago

Summing up:

Alchemy API key is now passed as ENV ALCHEMY_API_KEY
We will run e2e tests on tag creation in GitHub (not part of that commit)

MTB keys - we can store them in repo from the security perspective but they are too big :(

remote: error: File e2e_tests/docker-compose/keys/deletion_b10t30.ps is 143.42 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File e2e_tests/docker-compose/keys/insertion_b10t30.ps is 250.49 MB; this exceeds GitHub's file size limit of 100.00 MB

Any idea how and where we can store them and retrieve easily for e2e testing? Can we run tests with lower tree depth?

piohei commented 2 months ago

Docker compose execution downloads MTB keys if they are not present.