search

pycroscopy / sidpy

Python utilities for storing, processing, and visualizing spectroscopic and imaging data
https://pycroscopy.github.io/sidpy/
MIT License
11 stars 14 forks source link

Tools for provenance tracking #15

Open ssomnath opened 4 years ago

ssomnath commented 4 years ago

We need to come up with a way to track provenance of data as it is processed or analyzed. pyUSID forced all Processes to write data into the same HDF5 file. Databases like DataFed have inherent provenance tracking capabilities. However, for those who do not want to use or do not have access to DataFed, we may need to build on the existing pyUSID methodology while reconciling the need to track analyses that may have multiple input datasets - something pyUSID did not really solve all that elegantly.

ssomnath commented 4 years ago

Some legacy functions from pyUSID.io.hdf.simple.py have been ported to sidpy.hdf.prov_utils.py