search

ashish10alex / vscode-dataform-tools

Dataform tools - a vscode extension
https://marketplace.visualstudio.com/items?itemName=ashishalex.dataform-lsp-vscode
MIT License
20 stars 5 forks source link

Feature request: Search by tags in Dataform dependancy graph #36

Closed TheSmartMonkey closed 2 weeks ago

TheSmartMonkey commented 3 weeks ago

Summary

Be able to search by tags in Dataform dependancy graph

Motivation

See the full pipeline, useful when you have a lot of pipelines

Detailed Description

In dataform google ui

image

Add an option to search by tags

image

Expected Behavior

  1. Have an option to search by tags in dependencies graph (could be another dropdown or a third option in upstream / downstream)
  2. Could be cool to add a vscode command with ctrl + shift + P (on current file tags in dependancy graph and/or on all the tags)
ashish10alex commented 2 weeks ago

Hi @TheSmartMonkey , please try the latest release v0.2.14 that has the requested feature.

  1. User can selected a tag from the available tags from the web ui
  2. The options for the root node will be filtered to have only the nodes of the requested tag
  3. The tree is re-drawn by picking the first element from the filtered list. If you would like to see specific upstream / downstream dependency of a node for the the tag this can be achieved by selected it from the dropdown menu
  4. To go back to default view of all tags & nodes, you can select "all" option in tags dropdown. Now that I think of it, if someone names their tag as "all" it would not filter as expected

I have not implemented the keyboard shortcut for this feature as of now as it will be something that I think will be used scarcely. But can review this decision in future. Please let me know if it works for you :)

TheSmartMonkey commented 2 weeks ago

Hey @ashish10alex this is great man ! The only thing that could be improved from what I see is when I do a search by tag pretty mush all the time I want the full pipeline and not upstream / downstream

Currently : image

Full pipeline : image

This is the example of a simple pipeline but when I have assertions I could have a lot of dependencies and clicking on each nodes to display the full pipeline could be painful

For your select "all" option in tags dropdown issue I didn't look at your code but you can maybe display "all" in the frontend but in the backend do a match to some kind of tag with a unique id

ashish10alex commented 2 weeks ago

Hi @TheSmartMonkey , Please update to the latest version v0.2.17 . I have added two buttons expand all and collapse all which should easily let you expand the tree. However for seeing the entire graph, I am largely limited by the library I am using. Although I have been able to make some changes to the library by add colours for different datasets. Changing it to show the whole graph is a non-trivial change.

The current implementation servers most use cases for me, e.g. seeing the dependents and dependencies of a node easily, identify sources associated with the nodes and traversing to the file where source is defined by clicking on it.

However, there is definitely scope for improvement, so please feel free to submit a PR :)