Coveralls coverage

[MilesCranmer]

This is an announcement, not an issue.

Bifrost now has unit test coverage:

It's built into travis, so we won't be able to see test coverage for GPU-only modules.

On the coveralls page, there is a nice file-by-file listing of where greater test coverage is needed, and where we have hit 100% coverage.

[MilesCranmer]

It should also post coverage changes on each new pull request.

[benbarsdell]

This is great Miles!

Does it just include the Python files? 60% is actually higher than I expected given that it doesn't include GPU tests.

[MilesCranmer]

Yes, it's just the python files. Apparently there is a way to add it for C++ as well, but I assume that is for C++ tests, not something where they get linked with ctypes to C++ (I think we need some Google Tests). I could be wrong though. It would be great to see that coverage as well.

https://gronlier.fr/blog/2015/01/adding-code-coverage-to-your-c-project/

Also, I just realized there will also be a line-by-line highlighting of each file once I fix the paths (so they highlight where you aren't testing). More on this shortly.

[MilesCranmer]

Also, a lot of that coverage score could be coming from the old API: https://coveralls.io/builds/12262188/source?filename=%2Fusr%2Flocal%2Flib%2Fpython2.7%2Fdist-packages%2FBifrost-0.6.0-py2.7.egg%2Fbifrost%2Fblock.py

[MilesCranmer]

so they highlight where you aren't testing 

This sounds bad, sorry. I meant where the user isn't testing.

[MilesCranmer]

Okay, I have modified the relative paths on coveralls, and now each file shows line-by-line statistics on where the tests hit and where they don't.

[MilesCranmer]

Also, a lot of the test coverage points come from just hitting the function names without hitting their internals, see, e.g., 30% on this non-tested block: https://coveralls.io/builds/12262188/source?filename=%2Fusr%2Flocal%2Flib%2Fpython2.7%2Fdist-packages%2FBifrost-0.6.0-py2.7.egg%2Fbifrost%2Fblocks%2Fbinary_io.py

[MilesCranmer]

Coveralls apparently does not support multi-language reports: https://github.com/lemurheavy/coveralls-public/issues/544

However, codecov does: https://docs.codecov.io/v4.3.0/docs/merging-reports

[MilesCranmer]

So with Codecov, C++ coverage may be fairly simple: https://github.com/codecov/example-c

It shouldn't be difficult to switch to it, either.

[MilesCranmer]

Codecov working for python: https://codecov.io/gh/ledatelescope/bifrost