Closed Dr0p42 closed 1 year ago
@Dr0p42 seems like the PR is not passing, did you see?
SonarCloud Quality Gate failed.
1 Bug
0 Vulnerabilities
0 Security Hotspots
11 Code Smells
No Coverage information
0.0% Duplication
Let's merge @Dr0p42 ? Sonarcloud is still not happy
This pull requests resolves: https://github.com/jupyter-naas/naas/issues/344
It's adding logic to easily create pipeline on naas.
It's that easy:
This will give you the following
And after running it:
We can also run it by using
pipeline.run('progress')
which will show something like this:In this example we have a notebook failing on purpose to trigger the
on_error
step.Some other examples:
Imagine you have a bunch of notebooks to download videos from Youtube and compute a transcript out of it:
Here I am using DummyStep to give an idea, but it could definitely be
NotebookSteps
Now imagine you want to fetch a bunch of ticker/stocks and do some computation with it and ultimately buy or sell. You could fetch the ticker at the same time and also extract sentiment from twitter about the companies.
I hope that with these examples you are getting the idea. For now we have a bunch of Steps:
But we could definitely have more types of steps like: