Feature Request - run options (select multiple files, full refresh)

[mazum21]

Hey @ashish10alex

Two features for the dataform run command palette I would love to see:

1. full refresh option (for all other run option, e.g. run current file with dependents full refresh)
2. run multiple files at once (don't know how this could be implemented though)

Cheers Mario

[ashish10alex]

Hi @mazum21 , The ability to run multiple files is now available in v0.0.49. To run multiple files:

1. Open VS Code command pallet and run Dataform: Run multiple files
2. That will give you a list of .sqlx files where you can select multiple files and click ok
3. Once you click ok. The extension will determine the set of actions it needs to run and will run it in the terminal

---

full refresh option - I am still thinking on how to best provide this option to the user. Please let me know if you have any ideas / preferences

[mazum21]

Maybe reduce the run options in the command palette, e.g. only offer "Run current file","Run multiple files" and "Run one or more tags". All the other options could be specified in a dialogue. Similar to how it's done in the Dataform UI:

For the first tow options (current and multiple files) let the user select what should be run [] with dependants [] with dependencies [] full refresh [] only certain tags -> for "only certain tags" let the user select tags in the next step

For the third option (run one or more tags), let the user select one or more tags in the dialog.

What do you think?

[ashish10alex]

Hi @mazum21 , thanks for the input. If I try to copy the design pattern of Dataform UI. I will have to implement the run functionality in the side web view panel. So the flow in that case would be

1. Pick the file(s) / tag(s) you want to run from the picker ( ctrl/cmd + p )
2. Side web panel opens which lets your choose options such as include dependents / dependencies / full refresh etc.
3. Click run button

Although this is comprehensive, It is slower workflow and might be clunky to build into vscode extension. Most of the time I am running the file wt dependencies / dependents and currently my workflow for that is command pallet and select the option and enter ( all keyboard ).

Maybe I might convince myself to build the web UI version in sidepanel, but not sure atm. Also, Im unsure there is way to have stages when choosing things in command pallet picker. But please please feel free to send in a pull request Im happy to review the feature :)

[mazum21]

Yeah, I agree that the current workflow is very fast and I like it a lot. Still it would be nice to have more options and to be able to combine options. As far as I know it is possible to have multi stage in the command palette. It would still be fast to first choose between "Run current file","Run multiple files" and "Run one or more tags" and in the next stage choose the options. For most part you wouldn't have to choose an option and just press enter twice. But tbh I'm not sure whether this is really feasible. And sorry, I lack the skills to open a PR. Would love to to have them though.

[ashish10alex]

Hi @mazum21 , thanks for being persistent on this. Did some research seems like multi-stage in command pallet is certainly possible. Let me try to make an MVP for this feature request and get back to you.

[ashish10alex]

Hi @mazum21 , please try the latest version 0.0.57. The request feature has been made available via the command Dataform: Run file(s) / tag(s) with options For more details see https://github.com/ashish10alex/vscode-dataform-tools/pull/26

---

Let me know if this works for you :)

[mazum21]

Hey @ashish10alex thanks for the implementation. Looks good, but have to work with it to be able to give a better feedback :) you can close this issue if you want.