search

kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
https://kedro.org
Apache License 2.0
9.94k stars 903 forks source link

Save kedro pipeline image by pillow.ImageDataSet without manually launching kedro-viz server #375

Closed Minyus closed 4 years ago

Minyus commented 4 years ago

Description

Currently we have to launch the kedro-viz server to see the visualized pipeline. It will be more convenient if we can save the pipeline image by simply setting in catalog.yml like this:

_PIPELINE_IMAGE:
  type: pillow.ImageDataSet
  filepath: data/_pipeline_image.png

Context

This issue is related to #56 (moved to https://github.com/quantumblacklabs/kedro-viz/issues/31), but perhaps a more convenient feature.

Possible Implementation

Provide some of the visualization code in kedro-viz as a Kedro hook.

Possible Alternatives

Port some of the visualization code in kedro-viz to KedroContext.

lorenabalan commented 4 years ago

Hi @Minyus ! I'm a little apprehensive about using some magic in the catalog to get the pipeline visualisation. I wonder if there's a way to leverage @WaylonWalker 's kedro-static-viz plugin here?

Minyus commented 4 years ago

Hi @WaylonWalker , what do you think about using kedro-static-viz in a Kedro hook?

WaylonWalker commented 4 years ago

@Minyus You nearly read my mind, kedro-static-viz == 0.3.0 is out now with this functionality.

It does not create an image as the Issue is requesting, but it creates a static site will the same look, feel, and interactivity of kedro-viz, without the need to keep the flask server running if you have a place to host static content.

@lorenabalan Thanks for the plug here 😉. I saw the issue and didn't even think about kedro-static-viz as a solution.

Minyus commented 4 years ago

@WaylonWalker Awesome! I'll try it. Thanks!

lorenabalan commented 4 years ago

Closing this as resolved, loving the collaboration here. 😊