Apparently Joblib does a (so far so) good job at caching.
Additionally, it could be nice to be able to inspect what was cached & perhaps to reuse externally.
I am particularly referring to caching in csv files.
(Argument columns, return value column(s - for tuples))
Although it would only work well if the return data type is very fixed, such as: DataFrame or scalar, np.ndarray, yet these (i.e. a handful of) use cases seem to cover all I need.
Then they can be inspected, analyzed, etc. - and where I save results, reused.
Then as these results would go through a standard outputting mechanism, I could redirect the save paths depending on a specific scenario without wiring os.path.join()-s everywhere in the code, and also possibly making a natural distinction between input & output as opposed to an os.chdir().
But maybe there are just as good and less tricky options and/or pre-existing solutions.
Apparently Joblib does a (so far so) good job at caching.
Additionally, it could be nice to be able to inspect what was cached & perhaps to reuse externally.
I am particularly referring to caching in csv files.
(Argument columns, return value column(s - for tuples))
Although it would only work well if the return data type is very fixed, such as: DataFrame or scalar, np.ndarray, yet these (i.e. a handful of) use cases seem to cover all I need.
Then they can be inspected, analyzed, etc. - and where I save results, reused.
Then as these results would go through a standard outputting mechanism, I could redirect the save paths depending on a specific scenario without wiring os.path.join()-s everywhere in the code, and also possibly making a natural distinction between input & output as opposed to an os.chdir().
But maybe there are just as good and less tricky options and/or pre-existing solutions.