aws-samples / eks-kubeflow-workshop

Kubeflow workshop on EKS. Mainly focus on AWS integration examples. Please go check kubeflow website http://kubeflow.org for other examples
Apache License 2.0
96 stars 56 forks source link

Add more metadata tracking examples - notebook and pipeline #28

Open Jeffwan opened 4 years ago

Jeffwan commented 4 years ago
  1. Metadata by default is enabled for pipeline project. We want to give more examples to users.
  2. In other type of workloads, like model training, we want to add more examples (this has been added I think - double confirm or update the example if there's API changes).
PatrickXYS commented 4 years ago
  1. What type of Metadata examples we want to give? From the repo, I saw that we have this example. But for this example, we can only see the Metadata in the Artifacts store. In my opinion, we can add more metadata notebooks to bootstrap kubeflow pipeline. And our user should be able to see artifact visualization retrieved from every run's metadata.
  2. For other workloads, I found they have already been added to the repo, I'll sync with you and go head to confirm if they're compatible with kubeflow 1.0 currently.

WDYT @Jeffwan

PatrickXYS commented 4 years ago

From my part, I think current Metadata example is good enough, do you have any idea about other examples in metadata area?

I can give more examples on metadata, but it will be similar to the metadata example's Real World Example. Thus, I'm not sure if this is necessary. @Jeffwan

Jeffwan commented 4 years ago

Check my original asks, I think second one is ok. What about the first one, pipeline metadata tracking? @PatrickXYS

PatrickXYS commented 4 years ago

I'll be creating a real-world example with metadata artifacts visualization.

Jeffwan commented 4 years ago

@PatrickXYS for first item or second? If that's second, as we talked, we don't plan to add extra ones, right?

PatrickXYS commented 4 years ago

What I mean is write an example for lineage explorer tracking as we talked, wdyt? @Jeffwan

An example below: image