ME-ICA / aroma

ICA-AROMA, as a Python package. A work in progress.
Apache License 2.0
7 stars 11 forks source link

Add FSL image for testing #40

Closed vinferrer closed 3 years ago

vinferrer commented 3 years ago

Summary

I understand our objective is not u use FSL, however I think that at somepoint we need to setup a test that certifies our AROMA and FSL AROMA to be equivalent. Therefore I propose to find a docker image that can be able to run the integration test that we have now so later is easier to create this compatibility test.

-->

Next Steps

eurunuela commented 3 years ago

I do agree we should have a test to make sure we get the same results, at least until we translate the whole package into python.

tsalo commented 3 years ago

To clarify- are you saying that FSL should be run as part of the CI, or that we should use FSL to generate a set of long-term test files that will be used (but not generated) in CI?

eurunuela commented 3 years ago

I think Vicente is suggesting the former, but the latter sounds easier to me.

tsalo commented 3 years ago

I agree that a static set of results to compare against would be easier, and I am leaning in that direction (unless @vinferrer has a strong preference for running it in CI). I think it would be key, though, to have the actual code to run the FSL version stored in the repository, either as an example or as a skipped test. That way, we could re-run the code as needed.

eurunuela commented 3 years ago

That's a very good point. We could run all our tests on a docker image with FSL anyway, and so we can run the actual FSL code anytime as needed.

vinferrer commented 3 years ago

I agree that a static set of results to compare against would be easier, and I am leaning in that direction (unless @vinferrer has a strong preference for running it in CI). I think it would be key, though, to have the actual code to run the FSL version stored in the repository, either as an example or as a skipped test. That way, we could re-run the code as needed.

No strong preference actually, I was just thinking in some way of verifying that our results are equivalent to FSL aroma. The only problem I seem in having a static version is that we lost compatibility with newer versions of FSL though is not very likely

vinferrer commented 3 years ago

I agree that a static set of results to compare against would be easier, and I am leaning in that direction (unless @vinferrer has a strong preference for running it in CI). I think it would be key, though, to have the actual code to run the FSL version stored in the repository, either as an example or as a skipped test. That way, we could re-run the code as needed.

However, if the package already can run the FSL pipeline, why do you think is dfficult to run FSL aroma in the CI, there are fsl docker images: https://hub.docker.com/r/vistalab/fsl-v5.0, although is not the last version

eurunuela commented 3 years ago

However, if the package already can run the FSL pipeline, why do you think is dfficult to run FSL aroma in the CI, there are fsl docker images: https://hub.docker.com/r/vistalab/fsl-v5.0, although is not the last version

I guess the problem is not running FSL but the fact that we would be running the whole pipeline twice and it would be time-consuming.

vinferrer commented 3 years ago

Well they could be runned in parallel jobs and be checked at the end, or we could even setup the FSL one to be trigger when the FSL version changes, but for sure a static version is much easier

eurunuela commented 3 years ago

I've actually been thinking a bit about this and I think we should have an integration test with FSL until we are 100% python.

I've opened #41 to work on this.