kamu-data / kamu-cli

Next-generation decentralized data lakehouse and a multi-party stream processing network
https://kamu.dev
Other
303 stars 13 forks source link

kamu extra functions - request #38

Closed JvD007 closed 3 years ago

JvD007 commented 3 years ago

Is it possible to list all Workspace's created with kamu? And to use kamu to select the workspace and you want to use and jump to the correct folders? For example “kamu workspace list” and “kamu select workspace name

Other point: when you change the .yaml file, you need to delete the dataset first and the add and pull the dataset before you will the changes. Is it possible to create a “reload” function which will do this steps in one action (delete , add and pull the dataset).

This will speed up the data scientist his work.

Like “kamu reload xxxx.yaml

I hope this is possible :-)

sergiimk commented 3 years ago

Other point: when you change the .yaml file, you need to delete the dataset first and the add and pull the dataset before you will the changes. Is it possible to create a “reload” function which will do this steps in one action (delete , add and pull the dataset).

We have a lot of improvements planned around simplifying creation of the datasets including things like allowing to validate SQL query quickly and avoid re-downloading the data when iterating on a new dataset.

But for more convenience in the short-term I've added --replace flag, so:

kamu add --replace foo.yaml

will delete foo if it already exists and re-add it. This was released in v0.39.0.

Is it possible to list all Workspace's created with kamu? And to use kamu to select the workspace and you want to use and jump to the correct folders? For example “kamu workspace list” and “kamu select workspace name”

Personally I keep all my data projects in the same directory:

/home/me/projects/
  kamu_workspace_a/
  kamu_workspace_b/
  ...

so listing workspaces and switching between them from kamu would have no benefit compared to simple ls and cd.

In future we may revisit how workspces work in general, so I'll add this suggestion to our roadmap.