search

vliz-be-opsci / py-trav-harv

python module that will allow an enduser to perform link traversal on a triple store.
0 stars 0 forks source link

making a dump should be handled by __main__ cli, not part of travharv's `executor` #33

Closed marc-portier closed 2 months ago

marc-portier commented 3 months ago

https://github.com/vliz-be-opsci/py-trav-harv/blob/7088b65776926e73b42184e9b9e6af0e4ceed0c7/travharv/executor.py#L71

this looks like we are always producing a full dump, and we do it per task, but not tuned to the task? seems like something we can avoid (if no cli --dump is requested)

apparently only main is actually storing the output of this call, accumulating further to letting us handle that thing strictly in the main

(and similarly with the initial load in fact -- as suggested in #32 )