search

jhamrick / nbflow

A tool that supports one-button reproducible workflows with the Jupyter Notebook and Scons.
BSD 3-Clause "New" or "Revised" License
160 stars 18 forks source link

Support other build systems #1

Open jhamrick opened 8 years ago

jhamrick commented 8 years ago

Currently, nbflow only supports scons, but there's not really any reason why it can't in principle support other build systems. For example, I could support waf and then there would be python 3 support! :tada:

jhamrick commented 8 years ago

From playing around with this a little bit it seems like waf is nontrivial. But it might be possible to have it create a makefile or something and still get python 3 support.

belteshassar commented 7 years ago

Just watched your Scipy talk on Youtube. Great stuff!

Have you looked at Luigi? It is used a lot for building pipelines in the data science community.

jhamrick commented 7 years ago

I haven't looked at Luigi before -- thanks for the tip!

belteshassar commented 7 years ago

I thought a bit about this during the weekend. I don't think it's as easy as I first thought to adapt nbflow to Luigi. The problem is that in Luigi you have to specify the depencies as tasks rather than targets.

jhamrick commented 7 years ago

I haven't had a chance to look in detail at it yet, but skimming through the documentation it seems in principle something you should still be able to do... though I agree definitely a little trickier since it's all class-based.

On Wed, Nov 23, 2016 at 4:14 AM, Daniel Eriksson notifications@github.com wrote:

I thought a bit about this during the weekend. I don't think it's as easy as I first thought to adapt nbflow to Luigi. The problem is that in Luigi you have to specify the depencies as tasks rather than targets.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jhamrick/nbflow/issues/1#issuecomment-262498070, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFF9L4mRYF1YStptgU4gvXrp1lB_jxxks5rBC4YgaJpZM4H9KY5 .

twmr commented 6 years ago

Note that scons (>=3.0) already supports python3. Therefore, the README should be updated