Some duplicate functions have been replaced by the ALF functions in ONE
Filename pattern match now uses the Python3 fnmatch function
If a dataset type does not have a filename pattern a dataset is considered to be of that type if the object.attribute match the dataset type name
Regarding the last point: already most filename patterns reflect this anyway (e.g. *trials.choice.*). The advantage is that we can allow for datasets with arbitrary namespaces and timescales without risking ambiguity in the filename pattern, for example the dataset type trials.stimOn_times could have the following datasets: _ibl_trials.stimOn_times_bpod.csv and trials.stimOn_times.npy, while neither would match the dataset type passiveTrials.stimOn_times. The pattern *trials.stimOn_times*' would be too broad and would match passiveTrials.stimOn_times.npy for example.
Already the dataset type name must be unique in the database, however we would no longer require the filename pattern to be unique. This could be fixed with a little field type that converts empty strings to null (null != null, whereas '' == ''), however this would require a two or three step migration.
Changes made:
Regarding the last point: already most filename patterns reflect this anyway (e.g.
*trials.choice.*). The advantage is that we can allow for datasets with arbitrary namespaces and timescales without risking ambiguity in the filename pattern, for example the dataset typetrials.stimOn_timescould have the following datasets:_ibl_trials.stimOn_times_bpod.csvandtrials.stimOn_times.npy, while neither would match the dataset typepassiveTrials.stimOn_times. The pattern*trials.stimOn_times*'would be too broad and would matchpassiveTrials.stimOn_times.npyfor example.Already the dataset type name must be unique in the database, however we would no longer require the filename pattern to be unique. This could be fixed with a little field type that converts empty strings to null (null != null, whereas '' == ''), however this would require a two or three step migration.