search

aws-samples / eks-kubeflow-workshop

Kubeflow workshop on EKS. Mainly focus on AWS integration examples. Please go check kubeflow website http://kubeflow.org for other examples
Apache License 2.0
96 stars 56 forks source link

Reorganize Kubeflow pipeline examples #25

Open Jeffwan opened 4 years ago

Jeffwan commented 4 years ago
  1. Add one kubeflow native example using tfjob, etc
  2. Add taxi cab e2e example
  3. Add one cloud native example -> Use EMR to process data, Sagemaker to train model, Athena and then K8s to deploy the model?

Currently, we do have separate examples for EMR and Sagemaker. But we want to have one to demonstrate hybrid cloud concept.

Jeffwan commented 4 years ago

Rewrite kubeflow/examples mnist to a pipeline?

PatrickXYS commented 4 years ago

  1. According to the Kubeflow doc, we could rewrite this mnist example to a kubeflow pipeline.

  2. For taxi cab e2e example, currently it's blocked by the compatibility of TFX and Apache Beam 2.19.0 version. The details of the issue is here.

  3. Separate EMR and SageMaker example. Will figure out how to bring an e2e cloud native example.

Jeffwan commented 4 years ago

Hi, check this mnist example https://github.com/kubeflow/examples/tree/master/mnist

We want to have a similar one like this but probably convert to a pipeline like train job use tf-operator KFP component, etc.

This is not finalized