Filter out unpickleable objects before serialising session in kubeflow backend

SAME-Project / same-project

https://sameproject.ml/

Apache License 2.0

20 stars 8 forks source link

Filter out unpickleable objects before serialising session in kubeflow backend #97

Closed Bubblyworld closed 2 years ago

Bubblyworld commented 2 years ago

See #81.

While testing the roadsigns demo with multiple pipeline steps, some steps would fail because they contain unpickleable objects like csv readers. This change filters them out before dumping the session - might be worth logging the fields that were filtered so users aren't surprised by this.