First draft to change ando

[Slowblitz]

I've made some changes on how the AnDO works now it checks filenames only. There are still some changes to do in the generator tools and his related test. Now the dataset must contain this 3 files :

• dataset_descritption.sjon
• subject.tsv
• one datafile

Here's an example of a dataset :

[SylvainTakerkart]

Hi!

As we just discussed, two major things need to be changed from what you describe:

1. no check should be performed on the session id

2. the check on the directories and files should be performed in two steps: a. check the names of the directories (using the same algo as in the previous AnDO specs) b. then, check the names of the files

[Slowblitz]

So now the ando is working as we discuss in 2 steps first it checks the folder structure and then the filenames, I have also added some test , and worked on the structure of the project. There is still some work to do on the generator tools and the documentation

[SylvainTakerkart]

thanks, I will test and review before 3:30pm... you can switch to the development of the Generator until then ;)

[SylvainTakerkart]

[I removed my previous comment, I'd made a mistake; here is some new feedback]

when I run the checker on dataset004, I get several errors:

1.

` Level 3 error [session folder] at : ses-20200101_001_rap It should follow the ses-YYYYMMDD_XXX_BBBB format, where:

• ‘ses-’ is an imposed prefix
• ‘YYYYMMDD’ is the date of the session (8 digits, for instance 20180430 for April 30, 2018)
• XXX is the number of the session acquired on that date (3 digits, for instance 001 for the first session)
• BBBB is a string freely usable by the research group / user (is a string freely usable by the research group / user (for instance to add extra info on the version of the experimental protocol, on the type of preparation, on the user-friendly name of the animal etc.); this string cannot contain the underscore character. `

This seems to be an old error message that should not be there anymore...

2. ` Level 3 error [Ephys folder] at : ses-20200101_001_pop Ephys folder is missing or does not follow the AnDO specification rules:

• Ephys `

I think it should be Level 4 and not Level 3...

[SylvainTakerkart]

Hi! There are a lot of things to correct in the code, a bit everywhere, so that it would look nice... But maybe you can do this later... First, @Slowblitz , can you make it work as it should (i.e correct for what I said in my previous comment)?

@JuliaSprenger WDYT?

[SylvainTakerkart]

OK, after having been in the code for a bit of time now, let me try to make a checklist of what's do be done for this PR... At this point, I might have missed some things...

TODO list

• [x] Update error messages to match the current specs
• [ ] Make verbosity work for the checking at the file level (for now, I think that when an error is raised at the file level, no messages is printed out in verbose mode)
• [ ] Create datasets to test all the new errors
• [ ] Check whether the main algorithm (imbricated if) to combine the directory checking and file checking (is_AnDO function in engine.py) is actually what we want, and is the most computationally efficient

[SylvainTakerkart]

Not ready for reviewing yet... I will let you know!

[SylvainTakerkart]

OK, I'm starting to understand how the code works, but I think there are several mistakes which make it to complicated for me to fix it... I will focus on adding datasets to test the new errors (this should be my next commit), which can serve as a basis to test the corrections to be made in the code...

[JuliaSprenger]

I think we can close this PR as it an alternative proposal was merged with #63