search

ProjectQ-Framework / FermiLib

FermiLib: Open source software for analyzing fermionic quantum simulation algorithms
https://projectq.ch/
Apache License 2.0
87 stars 39 forks source link

Increase test coverage to 100% #9

Open babbush opened 7 years ago

babbush commented 7 years ago

We should aim for 100% test coverage. Coveralls reveals which modules have the least coverage; those with the least coverage, such as molecular_data should be improved first.

idk3 commented 7 years ago

See also https://coveralls.io/github/ProjectQ-Framework/FermiLib?branch=develop - our badge will update to 99%.

damiansteiger commented 7 years ago

Thanks for the new tests. It would be nice to get over 99.5% so the badge will show 100% ;-)

Also, currently we are reporting test coverage for unit tests and integration tests together. The functional tests inside src/fermilib/tests/ are great, however, we should maybe consider to anyway cover 99.5% using only unit tests as it is easier to pinpoint errors when unit tests fail...

I could remove the functional tests from counting towards tests coverage. What do people think? (It would still run them and report failures)

babbush commented 7 years ago

Hi Damian,

100% is an excellent goal, and one I'd like to achieve prior to our post-alpha launch. Ian is certainly being very helpful towards that goal! I understand where you are coming from with wanting us to cover the code with just unit tests but unfortunately there are parts of the code that would be very, very challenging (and/or tedious) to write meaningful unit tests for (as opposed to integration tests). A good example of this is the code which computes a molecular active space. Without being able to generate molecular Hamiltonians in the first place, one would need to contrive an example and work it out by hand to show this code works. It could be done but it would be very tedious and error prone. By contrast, by using real molecular Hamiltonians we can easily confirm that the right thing is happening. So at this point, I think it should be okay to allow the integration tests to cover certain parts of the code. That is my vote anyways but I am open to other people's opinions. I might add that I don't see this (use of integration instead of unit tests in new code) being abused much, if at all.