csv file sanity check

[aremazeilles]

I am updating (hopefully last version) the CI process, and I added some tools to check the format of the yaml and csv files.

As you can see in the CI process here, the check on the csv file is not passing on file ./tests/data/protocol3/input/platformData.csv:

Processing ./tests/data/protocol3/input/platformData.csv
Record #4649 has error: wrong number of fields in line
Record #7978 has error: wrong number of fields in line
Record #8838 has error: wrong number of fields in line
Record #13933 has error: wrong number of fields in line
Record #14245 has error: wrong number of fields in line
Record #14601 has error: wrong number of fields in line
Record #14973 has error: wrong number of fields in line
Record #15885 has error: wrong number of fields in line

I had a brief look at the file, and it seems that on the indicated lines, there are some extra column filled. Could you confirm that we can erase these extra values (I can do it if you want, if you confirm that I can).

Applying it locally, I could also detect issues into other files, that are:

Processing ./tests/data/protocol2/input/jointAngles.csv
file is valid
Processing ./tests/data/protocol2/input/platformData.csv
Record #4649 has error: wrong number of fields in line
Record #8838 has error: wrong number of fields in line
Record #13933 has error: wrong number of fields in line
Record #14245 has error: wrong number of fields in line
Record #14601 has error: wrong number of fields in line
Record #15885 has error: wrong number of fields in line
Processing ./tests/data/protocol2/input/emg.csv
file is valid
Processing ./tests/data/protocol7/input/jointAngles.csv
file is valid
Processing ./tests/data/protocol7/input/platformData.csv
Record #4649 has error: wrong number of fields in line
Record #14245 has error: wrong number of fields in line
Processing ./tests/data/protocol1/input/jointAngles.csv
file is valid
Processing ./tests/data/protocol1/input/platformData.csv
Record #8838 has error: wrong number of fields in line
Record #13933 has error: wrong number of fields in line
Record #14601 has error: wrong number of fields in line
Record #15885 has error: wrong number of fields in line
Processing ./tests/data/protocol1/input/emg.csv
file is valid
Processing ./tests/data/protocol3/input/platformData.csv
Record #4649 has error: wrong number of fields in line
Record #7978 has error: wrong number of fields in line
Record #8838 has error: wrong number of fields in line
Record #13933 has error: wrong number of fields in line
Record #14245 has error: wrong number of fields in line
Record #14601 has error: wrong number of fields in line
Record #14973 has error: wrong number of fields in line
Record #15885 has error: wrong number of fields in line
Processing ./tests/data/protocol6/input/jointAngles.csv
file is valid
Processing ./tests/data/protocol6/input/platformData.csv
Record #4649 has error: wrong number of fields in line
Record #14245 has error: wrong number of fields in line
Processing ./tests/data/protocol4/input/platformData.csv
Record #4649 has error: wrong number of fields in line
Record #7978 has error: wrong number of fields in line
Record #8838 has error: wrong number of fields in line
Record #13933 has error: wrong number of fields in line
Record #14245 has error: wrong number of fields in line
Record #14601 has error: wrong number of fields in line
Record #14973 has error: wrong number of fields in line
Record #15885 has error: wrong number of fields in line
Processing ./tests/data/protocol5/input/jointAngles.csv
file is valid
Processing ./tests/data/protocol5/input/platformData.csv
Record #4649 has error: wrong number of fields in line
Record #14245 has error: wrong number of fields in line

[juritaborri]

Sorry for the delay. Yes you can remove them

[aremazeilles]

I though I asked you, but it seems I did not. Could you handle it? It should be like removing the column directly

[juritaborri]

I found the problem, in some lines there are a different numeric format. I have modified it and create a new branch. I also add a function named as fillgaps that performs a spline interpolation in case of NaN in the experimental data. Can you confirm me that the function in the new branch are ok? @aremazeilles

[aremazeilles]

In #38 I corrected the reference files (you merged it I think), and effectively the issue was on the format of some cells.

Regarding the branch you mention , I would suggest you to do a Pull Request:

This way th CI process will verify the code, checking that calling your program with the reference input, we get the reference output. Then we can check the outcomes

[aremazeilles]

This can be closed, right?

[juritaborri]

Yes