Open ssomnath opened 4 years ago
Perhaps we may not need a Writer
class. We can make do with a kwarg: format
=nsid
or something like that
After further consideration. Provenance will be facilitated on an opt-in basis. Priority level for this issue has also been lowered. Possible solutions:
The problem with the first approach is that developers may not want to edit every function of theirs to add the kwargs and decorator. The problem with the second approach is that the end user would need to pass the arguments, keyword arguments, and outputs again to this writer class. Either way, someone will need to pay the price and there does not appear to be a simple enough solution.
We will need a very lightweight abstract class that standardizes the API for algorithms. This class will not include code for supporting embarrassingly parallel computation like
pyUSID.Process
does. However, it should have awrite_to_disk(writer)
wherewriter
is aWriter
object that needs to be implemented bypyNSID
andpyUSID
. The idea is that algorithms which are functions right now would be converted to / used in light-weight classes. Whenever a user uses a Process, the computation will be performed and returned in memory only. If the user wants, they could choose to write the results to disk and capture all the provenance.write_results()
will be the only thing that developers need to implement to enable provenance tracking in the HDF5 file