Code coverage?

[tueda]

How about adding test code coverage? Coveralls or Codecov.

From them, for example, you can see the polynomial routines (poly.cc) is relatively well checked by the continuous integration while the gzip routines (compress.c) is not.

A problem would be that the low percentage of the code coverage may discourage people from using FORM and it is difficult to increase the percentage on the release of v4.2...

[jodavies]

I started, but didn't get very far with, such an analysis.

In principle one could write code that writes form code to test all possible (valid) statement combinations, with arguments of numbers, dollars, expressions, etc etc.

Also scripts to test all possible sorting scenarios, by suitably setting the buffer sizes, termsinsmall, largepatches etc. Then one can try to identify parameters to reproduce issue #95 with a short runtime (currently the only example of this crash takes a few minutes to run -- no good for the CI suite).

[tueda]

For now, I just put --enable-coverage (also --enable-profile) option to the configure script (23e7afca67c26b9b7e93a588dedc699f905d056c). So, we can see the code coverage locally by using gcov or lcov. I'm still not sure whether Coveralls/Codecov would be useful for FORM development, and connecting to them may need Jos's help.

[vermaseren]

That sounds like something we can discuss on monday. I still have no idea what this coverage is about. But if it is useful, why not.

Jos

On 24 jun. 2017, at 17:07, Takahiro Ueda notifications@github.com wrote:

For now, I just put --enable-coverage (also --enable-profile) option to the configure script (23e7afc https://github.com/vermaseren/form/commit/23e7afca67c26b9b7e93a588dedc699f905d056c). So, we can see the code coverage locally by using gcov or lcov. I'm still not sure whether Coveralls/Codecov would be useful for FORM development, and connecting to them may need Jos's help.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/203#issuecomment-310843933, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxEsXPQrb54EzFWDvv6SiobJit72-Lks5sHSZCgaJpZM4ODca2.

[tueda]

Hi Jos,

See, for example, https://codecov.io/gh/tueda/form/src/codecov/sources/sort.c. It shows which lines are executed and how many times, during the running of the test suite (with FORM and TFORM). Red lines were never executed in the test cases, so in some sense one would say they are minefields, though off course the test cases are at the FORM language level and not in the C and so they are never perfect.

[tueda]

Now it seems to work with Coveralls, including ParFORM: https://coveralls.io/github/vermaseren/form.