Open marrrcin opened 2 years ago
This feature is really a must have to have a functional development environment with kedro.
@adrienpl Agree! The current development cycle with docker images is so painful...
Actually, I've been working on some implementation of this feature a year ago. Once I find the local branch, I will push it so somebody can take it over (I'm not into kedro/vertex anymore).
Thank you ! It will be really helpfull. What are you using right now ?
@adrienpl I'm no longer on a project in MLOps area, so I'm using just boring dev tools ATM ;-)
This idea is borrowed from Azure ML (and this PR https://github.com/getindata/kedro-azureml/pull/15 ) - where you define an
Environment
, which is a docker image which runs your image, but the code is not part of the image (only dependencies are present in the image). The workflow for that will make Data Science iterations faster, as they will not have to build the docker image every time they want to run / debug something in Vertex AI. This issue itself will be partially addressed by #81 , but this would be a next iteration on that.General workflow would work like this:
kedro vertexai run-once
with some flag (or maybe we should havekedro vertexai run
for docker andkedro vertexai run-once
for this flow 💡)Please discuss the design with @em-pe and @szczeles before implemeting.