i2mint / py2store

Tools to create simple and consistent interfaces to complicated and varied data sources.
MIT License
11 stars 2 forks source link

Declarative data format translation #50

Open thorwhalen opened 4 years ago

thorwhalen commented 4 years ago

A specification of a format-to-format that is translated to a conversion function with whatever tools are found locally.

For example, two rules could be {".wav": np.array, ".xls": pandas.DataFrame}, which is short hand for "source is a string path of a file with extension... if .wav, then we want a numpy array of the waveform, and if it's an .xls, we want a pandas DataFrame. But there could be several ways to get from the source to the required target, and there should also be (in the specification) rules for what to try first, and if the required module is not find, what to try second, etc. Possibly too, what to do if nothing is found (error, warn, ignore silently?)

A default of this specification should be used in QuickStores so as to service the "don't bother me with details, just try to figure it out on your own" use case.

Caveats (such as consistency between systems that have different libs installed) should be considered and handled appropriately.