search

cgat-developers / ruffus

CGAT-ruffus is a lightweight python module for running computational pipelines
MIT License
173 stars 34 forks source link

Exclude non-pickable arguments from checkpointing #50

Open ghost opened 9 years ago

ghost commented 9 years ago

Currently, non-pickable objects passed as (extra) argument to a task function cause a failed checkpointing operation that is not made explicit to the user. There should be a warning and said argument simply excluded from calculating the checksums.

Jerry-Ma commented 8 years ago

I came across similar issue just now and have one thought: could ruffus uses dill instead the default pickle facility for pickling? dill works for things that are not normally picklable with cPickle