search

hackalog / easydata

A flexible template for doing reproducible data science in Python.
MIT License
108 stars 22 forks source link

It's not possible to define datasource parse functions in Jupyter Notebooks #100

Open tiberiuichim opened 5 years ago

tiberiuichim commented 5 years ago

The tutorial gives the impression that it would be possible to create a notebook that both defines the data source (and integrates it in the workflow) and demonstrate its usage. This is not possible if the datasource needs a transform function that's defined in the notebook because of serialization/deserialization problems.

hackalog commented 5 years ago

Can you point me at the spot you're referring to? In our tutorials, we do make note of this fact, (by bumping up against the problem and then solving it as an exercise) and suggest using %%writefile -a from within the notebook, (or reading and then writing localdata.py using similar magics). Maybe I forgot to update the tutorial somewhere?