georgymh / decentralized-ml

Interoperable and decentralized machine learning.
Apache License 2.0
9 stars 5 forks source link

Dataset mappings #35

Closed neeleshdodda44 closed 5 years ago

neeleshdodda44 commented 5 years ago

Hopefully, this a short PR.

Dataset Manager

Tl;dr the implementation mirrors that of the Configuration Manager (I ultimately decided not to just put the mappings in configuration.ini since I thought that would break abstractions).

NOTE: I accidentally committed parts of the first two methods above in a previous PR, please make sure to still review all of each of the methods.

neeleshdodda44 commented 5 years ago

@georgymh could you explain why UUID again? Like instead of just using the dataset names or other ways of generating random identifiers?

I have an idea of how to make this smoother, but I'm not sure if it would compromise your original reasons for having identifiers in the first place.

neeleshdodda44 commented 5 years ago

So the reason why I ask is because it might be better to use some kind of deterministic encoding (like uuid3/uuid5 and some arbitrary namespace in the docs) so that we don't even need to store the mappings in some file. Obviously not necessary (and there's some drawbacks), but I think it's alternative to consider.

Since I've got two approvals, I'm just going to go ahead and merge.