[BOUNTY] Community Improvement Bounty - libp2p - Conformance Test Kit

[raulk]

libp2p conformance test kit

Hackers, roll your sleeves up and hack for a chance to win 3000 EUR! 🤑

See devgrant 5 in the libp2p/devgrants tracker: https://github.com/libp2p/devgrants/issues/5

The specification can be found here: https://github.com/libp2p/devgrants/blob/master/005-conformance-test-kit.md.

How to qualify

We know you are an amazingly talented hacker, but also recognise that developing a full conformance test kit is not feasible in 2-3 hackathon days!

You would qualify if you develop the following components in Golang:

1. The simple text-based control protocol over TCP, as defined in the specs.
2. A modular framework into which test cases can be plugged.
   • Test cases should receive textual arguments (from control messages).
   • Each test case is identified/selected by an ordinal.
3. At least eight test cases covering protocol selection, SecIO (as a secure channel) and mplex (as the simplest multiplexer). Each test case should print PASS/FAIL.
4. A pull request to go-libp2p adding a test case that instantiates a libp2p host, and subjects it to the conformance test kit.

If you intend to work on this, speak to @raulk before you start for guidance and scope setting!

Resources

See https://github.com/ethberlinzwei/KnowledgeBase/blob/master/resources/libp2p.md, and use @raulk as a walking encyclopedia for all things libp2p..

Judging Criteria

1. Completeness and maintainability of code.
2. Easiness of plugging in new test cases.
3. Reporting on PASS/FAIL, especially important on failures (why did a test case fail?).
4. Documentation / code comments.

Prizes

1. 1000 EUR via the Community Improvement Bounties fund.
2. 1000 EUR via Protocol Labs.
3. 1000 EUR via the Ethereum Foundation.

[mariapaulafn]

Note - The Community Improvement Bounties are funded by the ETHBerlin fundraising efforts - if your project needs these bounties to work better with libp2p - consider helping out. High incentives make happy hackers

[djrtwo]

The Ethereum Foundation is chipping in 1000 EUR for this bounty!

[samparsky]

@raulk This is like a battery of integration tests against a libp2p node client. Could you explain more on simple text-based control protocol over TCP?

My thought process is something around a client can choose using a flag the set of integration tests to run against its implementation. i.e the integration tests are broken down into segements like secio, mplex.

I would imagine starting a CTK with the below command

./ctk --tests="secio,mpex"

Also, do you have telegram I can reach you on?

[samparsky]

@raulk Hello, how is it possible to reach out to you or who could be of help on this issue?

[raulk]

@samparsky Hey Sam, glad to have met you in person at ETHBerlin, and that you're excited about contributing to libp2p! 🙌

My Telegram is my github user with an r at the end -- feel free to ping me there.

Could you explain more on simple text-based control protocol over TCP?

The rationale is that the implementation under test and the CTK will need to communicate to "switch" between cases, marking the end of a test case and the beginning of the next one.

At first I did think we might not need this explicit coordination, and that we might be able to run a deterministic battery of tests in a predetermined order, but this would be too fragile: one mistake and you send the whole battery of tests off track. Also, we need to cater for test cases initiated by the CTK, and test cases initiated by the implementation under test, and it's not easy to do that unless we have those two communicate to "set the stage".