search

jdtournier / testing

The old MRtrix3 testing repo, now obsolete - do not use!
0 stars 0 forks source link

tcksample needs appropriate testing_diff command #4

Open dchristiaens opened 8 years ago

dchristiaens commented 8 years ago

I'm looking for a way to compare tcksample outputs. Unix diff for text files can't deal with small numeric errors. Our own testing_diff_matrix requires matching row lengths, and therefore won't work either...

jdtournier commented 8 years ago

Good point. One possibility, although not perfect, is to use cat values.txt | tr ' ' '\n' > values_as_column.txt to force all the values out as a single column vector, which can then be used with testing_diff_matrix. You can also check that the number of lines & words in the output matches with cat values.txt | wc -l -w. If anything breaks, the chances of it not affecting these tests should be pretty low...

jdtournier commented 8 years ago

Other option of course, is to modify testing_diff_matrix to actually allow non-matching row sizes, in which case it's not really testing matrices of course... This would require slightly different handling, as we couldn't rely on the MRtrix matrix loading functionality.

dchristiaens commented 8 years ago

Smart to put them all in one vector. I thought about iterating through all lines of both files and running testing_diff_matrix on each of them, but it's a bit tricky to get in on a single line for the test script to recognise it.

Modifying the testing_diff_matrix is neat. We could include an option -check_dims to enforce a separate check on the dimensions when providing a matrix. (Or perhaps the other way around to make matrices the default).

jdtournier commented 8 years ago

I'm thinking modifying testing_diff_matrix is probably the simplest and cleanest option. We don't need to add a check for dimensions, since the test would trivially fail if the ground truth file is itself a matrix...