search

xcompact3d / x3d2

https://xcompact3d.github.io/x3d2
BSD 3-Clause "New" or "Revised" License
3 stars 4 forks source link

Investigate using pfUnit as formal testing framework #36

Closed JamieJQuinn closed 6 months ago

JamieJQuinn commented 7 months ago

Currently unit tests are defined as regular subroutines and run through ctest. We may get better utilities using pfunit, like better float comparison, test tagging, running tests in parallel.

@slaizet said Thibault tried this out last year and described it as "painful" so I'd like to know what benefits we might get from pfunit before ripping out our current working test framework.

Nanoseb commented 7 months ago

I've used pfUnit a bit in the past, but never done the setup from scratch so I am not sure how painful that part is. I think the main advantage is being able to cleanly run test with MPI. ctest can kind of do it (#20), but it seems fairly hacky (nevermind, it seems to work well).

An other option would be to use something like lit (https://llvm.org/docs/CommandGuide/lit.html). We are using it with Paul on an other fortran project and it is fairly straightforward.

pbartholomew08 commented 7 months ago

I've used pFUnit in the past and found it a pain to work with, I would recommend ctest over lit.

JamieJQuinn commented 6 months ago

@pbartholomew08 we're currently using ctest!

Considering many folks' poor experiences with pfunit, one fewer dependency to worry about, potential issues with f2008 + Nvidia's compilers, and the fact that ctest seems to be working fine for us now, I'm considering pfunit unsuitable.