kedro-org / kedro-viz

Visualise your Kedro data and machine-learning pipelines and track your experiments.
https://demo.kedro.org
Apache License 2.0
677 stars 112 forks source link

Clarify `--save-file` #1681

Open astrojuanlu opened 10 months ago

astrojuanlu commented 10 months ago

Description

I was just trying the --save-file option and discovered that it produces a directory, rather than a file:

$ kedro viz --save-file viz.json --env demo
...
Kedro Viz Backend Server started successfully...

^C
$ tree viz.json
viz.json
└── api
    ├── main
    ├── nodes
    │   ├── 00bf9601
    │   ├── 080067ed

Expected Result

A file was saved, as explained in the docs https://docs.kedro.org/projects/kedro-viz/en/latest/kedro-viz_visualisation.html#share-a-pipeline-visualisation

Actual Result

A directory was created.

Your Environment

I used Kedro Viz 6.7.0.

Checklist

astrojuanlu commented 10 months ago

By the way, I tried --load-file viz.json immediately after that and I got an ugly

                             FileNotFoundError: [Errno 2] No such file or directory:                                                    
                             '/Users/juan_cano/Projects/QuantumBlack Labs/talk-kedro-huggingface/output.json/main'

turns out I had to do --load-file viz.json/api instead.

ravi-kumar-pilla commented 10 months ago

Hi @astrojuanlu , Thank you for raising this. We have addressed a part of the issue here. But we plan to do a refactor on kedro viz run --save-file and --load-file options in coming sprints. Thank you

rashidakanchwala commented 9 months ago

This is done.

astrojuanlu commented 9 months ago

I just tested --save-file with Kedro-Viz 7.0.0 and I still get the same: a directory is created instead of a file.

What has changed @rashidakanchwala ? Am I doing something wrong?

rashidakanchwala commented 9 months ago

hi Juanlu, that's correct. Earlier --save-file and --load-file only saved the main pipeline json which is the default pipeline flowchart. If you went to click on any nodes or switch pipeline from dropdown view nothing would work because that information is under nodes and pipeline folder. We recently fixed this so now when you do --save-file a directory is created which has main.json (for default pipeline) a pipeline folder with other pipeline.json and nodes folder which has metadata of all nodes.

rashidakanchwala commented 9 months ago

Having said that, we need to update this in the docs

This command will save a visualisation of the __default__ pipeline as a JSON file called my_shareable_pipeline.json. It doesn’t share data, such as that in the code panel, nor can you share images or charts.

i had missed this. Thanks for flagging

astrojuanlu commented 6 months ago

I think we didn't quite address this in a satisfactory way. I'm reopening.

astrojuanlu commented 6 months ago
$ find exportation2/ -name "*.json"
$

there are no JSON files in the result of --save-file

astrojuanlu commented 6 months ago

TIL: The JSON file is there, it's just that it doesn't have a .json extension:

❯ head exportation/api/main 
{
  "nodes": [
    {
      "id": "d75ec4b0",
      "name": "create_confusion_matrix",
      "tags": [],
      "pipelines": [
        "reporting",
        "__default__"
      ],
...

This is less than ideal. It's might look like a petty thing but is there a chance to rename the file?

rashidakanchwala commented 6 months ago

I guess we could -- But I think in general users usually don't interact directly with the files. They load it directly using --load-file, and in the javascript code, it reads it this way ... because even the json content of this file won't make sense to them.

astrojuanlu commented 3 months ago

Could be addressed with #1705 ?