Validator Stack: Full Integration test suite

Ivshti commented 5 years ago

Create an integration test suite for the AdEx validator stack

Protocol spec: https://github.com/AdExNetwork/adex-protocol

Validator stack implementation: https://github.com/AdExNetwork/adex-validator-stack-js

OUTPACE: https://github.com/AdExNetwork/adex-protocol/blob/master/OUTPACE.md

a bunch of integration tests need to be written, including a group that tests inter-validator relationships and possible attacks: e.g. leader signs an invalid state, and followers should detect and not sign

other ones:

follower should mark unhealthy, but not if the difference is too little
try denial of service attack vector

Testing the regular code paths should work in the following way:

create two environments (dockerized?) where each connects to it's own MongoDB database (could be connected to the same server, but at a different DB)
both DBs will be seeded: example in test/prep-db
each environment will run a validator stack: a sentry and a validator worker, initially with the dummy adapter
in the test data (in the DB), there should be a channel that appoints the first validator worker as a leader, and the other as a follower
start sending events to both sentries (example in test/send-events) and see how the system reacts
run integration tests for the sentry methods (e.g. getting info for a channel, detailed info for a channel, etc)

later, we'd like to have larger coverage, including the ethereum adapter and watcher

furthermore, integration tests of the adapters, individually, are important as well

Important note regarding Gitcoin: there is a gitcoin bounty for this issue: https://gitcoin.co/issue/AdExNetwork/adex-validator-stack-js/1/1996

This test suite is supposed to test one of the most critical components in AdEx (the validator stack), so we believe in creating our own in-house implementation as well as posting a bounty for a third-party implementation. Ideally, we will end up with two independent test suites for this critical component.

gitcoinbot commented 5 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

This issue now has a funding of 40.0 ETH (3422.6 USD @ $85.56/ETH) attached to it as part of the AdEx Network fund.

If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
- Want to chip in? Add your own contribution here.
- Questions? Checkout Gitcoin Help or the Gitcoin Slack
- $45,255.50 more funded OSS Work available on the Gitcoin Issue Explorer

samparsky commented 5 years ago

@Ivshti I am quite interested in working on this. I have gone through the OUTPACE specification and I don't have any questions yet

But looking the protocol implementation, is there any reason why there is no skin in the game e.g. staking for validators since they are essentially different from the participants?

Also how does "publisher-side platform(s) should be operated by consortiums of the largest publishers" address the 2/3 validators becoming malicious?

Also how is the validator leader chosen, is it randomly?

samparsky commented 5 years ago

@Ivshti Is this still available or its already being worked on?

Ivshti commented 5 years ago

@samparsky so sorry, I rejected your application by mistake and for some reason I haven't received a single notification from this thread

it's still available and we'd be happy if you start work on it (please re-submit a request)!

re the questions:

Staking and skin in the game exists but it's a part of a separate component: https://github.com/AdExNetwork/adex-protocol/issues/7 ; this only serves to help assign the validators though

re this: "Also how does "publisher-side platform(s) should be operated by consortiums of the largest publishers" address the 2/3 validators becoming malicious?": the basic assumption is that half of the validator voting power would be operated by publisher(s), the other half by advertiser(s); or in other words, by a publisher-side platform and an advertiser-side platform

opposing interests is what guarantees the OUTPACE security (as described in the protocol docs)

if the question is "what keeps delegated publisher-side platforms and advertiser-side platforms becoming malicious together and stealing user funds" - the answer is:

1) large users should operate one of those themselves (e.g. you'd assing your own publisher-side platform) 2) if they don't: https://github.com/AdExNetwork/adex-protocol/issues/7

samparsky commented 5 years ago

@Ivshti Awesome! Will submit another request right away

gitcoinbot commented 5 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work has been started.

These users each claimed they can complete the work by 4 days from now. Please review their action plans below:

1) samparsky has been approved to start work.

Will start work on it right away :D

Learn more on the Gitcoin Issue Details page.

Ivshti commented 5 years ago

@samparsky let me know if you have any further questions

I propose that we collaborate on the list of test cases: let's make a hackmd for it

samparsky commented 5 years ago

@Ivshti That would be awesome! Do you have a Gitter or Slack channel where we can chat?

Ivshti commented 5 years ago

@samparsky we have a discord, please ping me on email (ivo@adex.network) and i'll send you an invite

samparsky commented 5 years ago

@Ivshti Cool!, I have sent you an email

gitcoinbot commented 5 years ago

@samparsky Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

[x] reminder (3 days)
[ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

samparsky commented 5 years ago

@Ivshti You can checkout the document now https://hackmd.io/s/BJWSUZ5ZE

gitcoinbot commented 5 years ago

@samparsky Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

[x] reminder (3 days)
[ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

gitcoinbot commented 5 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work for 40.0 ETH (4936.65 USD @ $123.42/ETH) has been submitted by:

@samparsky

@Ivshti please take a look at the submitted work:

PR by @samparsky

Learn more on the Gitcoin Issue Details page
Want to chip in? Add your own contribution here.
Questions? Checkout Gitcoin Help or the Gitcoin Slack
$47,735.75 more funded OSS Work available on the Gitcoin Issue Explorer

gitcoinbot commented 5 years ago

⚡️ A *Javascript Master Scriber* Kudos has been sent to @samparsky for this issue from @Ivshti. ⚡️ The sender had the following public comments: > Great work on a complex task! Hope we continue collaborating in the future! Nice work @samparsky! Your Kudos has automatically been sent in the ETH address we have on file.

gitcoinbot commented 5 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

The funding of 40.0 ETH (4652.55 USD @ $116.31/ETH) attached to this issue has been approved & issued to @samparsky.

Learn more on the Gitcoin Issue Details page
Questions? Checkout Gitcoin Help or the Gitcoin Slack
$37,513.18 more funded OSS Work available on the Gitcoin Issue Explorer

AmbireTech / adex-validator

Validator Stack: Full Integration test suite #1