While trying to write the prognostic ML model code, I had a difficulty with installing extra python packages into the docker image. While the fv3run functions can take a runfile argument, there is no such argument for adding other python dependencies (e.g. a local module).
Currently, this has made developing with fv3run pretty complicated, since any changes require rebuilding a docker image which contains the desired package. I think @oliverwm1 ran into a similar issue today. I think one solution could be to add an extra_packages argument to the run_docker that could be pointed to a list of python_modules or packages to include in the run. This is pretty similar to how dataflow works.
While trying to write the prognostic ML model code, I had a difficulty with installing extra python packages into the docker image. While the
fv3run
functions can take a runfile argument, there is no such argument for adding other python dependencies (e.g. a local module).Currently, this has made developing with
fv3run
pretty complicated, since any changes require rebuilding a docker image which contains the desired package. I think @oliverwm1 ran into a similar issue today. I think one solution could be to add anextra_packages
argument to therun_docker
that could be pointed to a list of python_modules or packages to include in the run. This is pretty similar to howdataflow
works.