search

CONABIO / kube_sipecam_playground

Jupyter notebooks to test kube_sipecam deployments in k8s
MIT License
0 stars 0 forks source link

Process in parallel more than one ODC tile via kubeflow pipelines #12

Open palmoreck opened 4 years ago

palmoreck commented 4 years ago

Launch more than one ODC tile lc pipeline lc workflow as described in:

1_issue_8_pipeline_lc_MAD-Mex_pixel_wise_using_kale

for parallel execution.

Also this nbook can be modified according to https://github.com/CONABIO/kube_sipecam_playground/issues/11

palmoreck commented 4 years ago

Nbook 3_issue_12_pipeline_lc_MAD-Mex_pixel_wise_using_kale_and_distributed solves this issue. But it will be interesting launcing one pipeline per ODC tile. Could be useful for this:

titanicml-euyio.kale#L1112

titanic-ml-qmn2n.kale#L1044

titanic-ml-30-04-2020-wyom0.kale.py#L1016

And:

pipeline_lightweight

All of this refs use kubeflow pipelines in python3.

palmoreck commented 3 years ago

See https://github.com/CONABIO/kube_sipecam_pipelines/issues/1

And hsipipeline.py for an example that is useful