ploomber / soopervisor

☁️ Export Ploomber pipelines to Kubernetes (Argo), Airflow, AWS Batch, SLURM, and Kubeflow.
https://soopervisor.readthedocs.io
Apache License 2.0
45 stars 18 forks source link

soopervisor add/export are too verbose #69

Open edublancas opened 2 years ago

edublancas commented 2 years ago

to use soopervisor, we need to load the dag in a few places, each time we load it, we output some messages to the user, but this gets too much in some cases. I think most backends output the same messages twice, kubeflow makes it three times.

I think the best thing is to load once and pass the loaded dag, but if not possible, we can load a second time but suppress the output.