search

GSTT-CSC / MLOps

Framework for building ML apps
GNU General Public License v3.0
9 stars 5 forks source link

Split local and live volume mount #16

Closed laurencejackson closed 2 years ago

laurencejackson commented 3 years ago

To make testing easier it would be good if we can separate the docker mounts for local and remote volumes.

We should have both LOCAL_VOLUME_MOUNT and VOLUME_MOUNT values in the config file. Which one is used at runtime can be defined by the use_localhost flag.

hshuaib90 commented 3 years ago

What's the current challenge with testing?

laurencejackson commented 3 years ago

Currently having to manually set the data location in the code depending on if the developer is testing locally or running on DGX. It can be useful to have a small dataset locally just to check for runtime errors. It would be useful to be able to use the localhost flag to indicate that you want to use this dataset.

Could be clearer if we refactor the use_localhost flag to something like test

laurencejackson commented 2 years ago

This is resolved by #29 by the if use_localhost: condition in mlops.ProjectFile.init

So long as users are running their project through through the Experiment class then the correct volume mount is defined by the config file. The data directory will always be mounted to the internal location /DATA