detect and load additional metadata recursively

[lucasgautheron]

Description

Currently, recordings and children metadata are assumed to be entirely stored in metadata/recordings.csv and metadata/children.csv respectively. However, in case users spread their metadata across several files (e.g. to achieve multi-tier access), then there is no standard.

I suggest the standard should be that additional metadata belong as extra CSVs in metadata/recordings and metadata/children respectively. These folders are optional.

If the package detects them, it will list all CSVs from these folders recursively, sort them alphabetically (currently based on their basename, which might be ambiguous), then merge them sequentially.

For instance, if recordings/0.csv and recordings/1.csv exist, the package will :

• load metadata/recordings.csv as a dataframe
• load recordings/0.csv, merge it with the previous dataframe; in case of duplicate columns, recordings/0.csv will prevail
• load recordings/1.csv, merge it with the previous dataframe; in case of duplicate columns, recordings/1.csv will prevail

etc.

Status

• [x] tests
• [x] documentation
• [ ] implementation
  • [x] merge additional metadata recursively (alphabetical order)
  • [ ] improve tests so that they can identify problematic files... this is a big change.

[alecristia]

sort them alphabetically (currently based on their basename, which might be ambiguous)

I didn't see in your code a check for ambiguity (ie without path, the names are identical).

I may have also missed a set of messages for the user:

• multi-tier detected
• merging x and y (in that order of precedence)

and/or error:

• basenames are identical -- please disambiguate precedence for multi-tier metadata

(my messages are super cryptic, but you see the idea!)