Adding Tests + CI

[bhuvy2]

The CI can be mostly copied over from broadway-api, but it'd be nice to have some assurance checks on each commit.

[ayushr2]

I think this issue needs to be shifted to Chain Link. Broadway grader just communicates with the API and passes on the job to chainlink to run. @nmagerko wrote some tests for chain link which tests what we want to test.

What would we want to test on the broadway-grader side of things?

[bhuvy2]

There is also a discussion of whether we want to merge chain link. Assuming we want to keep the different, we should just sanity test heartbeating and whether chainlink calls the right funcs

[ayushr2]

Yep, I think we should keep them separate. Chainlink is meant to be a public python library for running a chain of containers. This project is more Broadway specific.

[bhuvy2]

I don't want to get into the whole debate of what warrants a package (I strongly side on functions or few functions don't warrant packages re: the left-pad dabacle, and chain-link in spirit is a different functionality, but is not a collection of functions or otherwise.

Either way, mocking should be added.

[nmagerko]

For testing, I think this project might need some refactoring first (this project can be tested in a way that is separate from how chainlink tests). We should at least be able to test that a job is submitted to some black box (e.g. chainlink mocked) to run, and that the results we get back from the broadway-grader routine that does that makes sense.

[nmagerko]

The actual execution of running containers in sequence should be tested more throughly, but that is not the responsibility of this project. I'd say that needs to be a chainlink issue.

[ayushr2]

As of now,

• We have added integration tests which run the api and grader together.
• Nick has refactored this project already.
• Chainlink has some tests.