Closed ca-scribner closed 3 years ago
Components get_from_minio
and put_to_minio
are complete. They:
A simple example of use is also included, and serves as a rudimentary test case.
The files are temporarily located off StatCan repos because I haven't found a natural home for them yet.
Future work:
cc @blairdrummond @brendangadd
To help users access files from MinIO when using kubeflow pipelines, create reusable components (eg: component yaml files that can be accessed via
load_component_from_url()
). These components should use the automatically mounted secrets for MinIO and avoid errors due to the race condition where the main pod may be running before vault has mounted the secrets.