Closed vinferrer closed 3 years ago
I do agree we should have a test to make sure we get the same results, at least until we translate the whole package into python.
To clarify- are you saying that FSL should be run as part of the CI, or that we should use FSL to generate a set of long-term test files that will be used (but not generated) in CI?
I think Vicente is suggesting the former, but the latter sounds easier to me.
I agree that a static set of results to compare against would be easier, and I am leaning in that direction (unless @vinferrer has a strong preference for running it in CI). I think it would be key, though, to have the actual code to run the FSL version stored in the repository, either as an example or as a skipped test. That way, we could re-run the code as needed.
That's a very good point. We could run all our tests on a docker image with FSL anyway, and so we can run the actual FSL code anytime as needed.
I agree that a static set of results to compare against would be easier, and I am leaning in that direction (unless @vinferrer has a strong preference for running it in CI). I think it would be key, though, to have the actual code to run the FSL version stored in the repository, either as an example or as a skipped test. That way, we could re-run the code as needed.
No strong preference actually, I was just thinking in some way of verifying that our results are equivalent to FSL aroma. The only problem I seem in having a static version is that we lost compatibility with newer versions of FSL though is not very likely
I agree that a static set of results to compare against would be easier, and I am leaning in that direction (unless @vinferrer has a strong preference for running it in CI). I think it would be key, though, to have the actual code to run the FSL version stored in the repository, either as an example or as a skipped test. That way, we could re-run the code as needed.
However, if the package already can run the FSL pipeline, why do you think is dfficult to run FSL aroma in the CI, there are fsl docker images: https://hub.docker.com/r/vistalab/fsl-v5.0, although is not the last version
However, if the package already can run the FSL pipeline, why do you think is dfficult to run FSL aroma in the CI, there are fsl docker images: https://hub.docker.com/r/vistalab/fsl-v5.0, although is not the last version
I guess the problem is not running FSL but the fact that we would be running the whole pipeline twice and it would be time-consuming.
Well they could be runned in parallel jobs and be checked at the end, or we could even setup the FSL one to be trigger when the FSL version changes, but for sure a static version is much easier
I've actually been thinking a bit about this and I think we should have an integration test with FSL until we are 100% python.
I've opened #41 to work on this.
Summary
I understand our objective is not u use
FSL
, however I think that at somepoint we need to setup a test that certifies our AROMA and FSL AROMA to be equivalent. Therefore I propose to find a docker image that can be able to run the integration test that we have now so later is easier to create this compatibility test.-->
Next Steps