search

databio / pypiper

Python toolkit for building restartable pipelines
http://pypiper.databio.org
BSD 2-Clause "Simplified" License
45 stars 9 forks source link

Should we remove checkpointing functionality? #184

Closed nsheff closed 1 year ago

nsheff commented 1 year ago

@vreuter at one point you started, but I think didn't finish, checkpointing functionality.

As far as I know, this has never been used in practice in one of our pipelines. Should we just remove it to simplify the code and make pypiper more maintainable? I am open to ideas.

vreuter commented 1 year ago

@nsheff

I'd prefer if it stays, as I'm using this quite a bit in at least 1 project at the moment. If you really want to remove it it, it's OK and I can maintain another fork. As is, for the moment I've pinned the version I'm using 0.12.3 to avoid pulling in some SQL-related dependencies through pipestat.

Anyway, all that to say I'm open to different logistics about how to have this functionality than keeping it part of the main package, but I would say that I use it quite a lot.

nsheff commented 1 year ago

I didn't realize you were actively using this. it's fine if it stays in that case!