Closed neeleshdodda44 closed 5 years ago
@georgymh could you explain why UUID again? Like instead of just using the dataset names or other ways of generating random identifiers?
I have an idea of how to make this smoother, but I'm not sure if it would compromise your original reasons for having identifiers in the first place.
So the reason why I ask is because it might be better to use some kind of deterministic encoding (like uuid3/uuid5 and some arbitrary namespace in the docs) so that we don't even need to store the mappings in some file. Obviously not necessary (and there's some drawbacks), but I think it's alternative to consider.
Since I've got two approvals, I'm just going to go ahead and merge.
Hopefully, this a short PR.
Dataset Manager
create_dataset_mappings
to generate mappings from random UUIDs to dataset folders, and store this in a yaml file calleddatasets.yaml
.bootstrap
to decide whether to load the yaml file (if it exists) or callcreate_dataset_mappings
get_mappings
that returns the loaded mappings, if they exist.Tl;dr the implementation mirrors that of the Configuration Manager (I ultimately decided not to just put the mappings in configuration.ini since I thought that would break abstractions).
NOTE: I accidentally committed parts of the first two methods above in a previous PR, please make sure to still review all of each of the methods.