search

USGS-R / drb-estuary-salinity-ml

Creative Commons Zero v1.0 Universal
0 stars 4 forks source link

add replicates and docstrings to run_model #101

Closed galengorski closed 1 year ago

galengorski commented 2 years ago

This pull request contains 3 files:

  1. run_model.py -- a series of data prep, and modeling functions
  2. model_config.yaml -- a config file for the modeling steps
  3. Snakefile_b_ml_model_baseline -- a snakemake file to run the steps of run_model.py and keep track of the inputs and outpus
amsnyder commented 2 years ago

When I am importing LSTMDA_torch.py, I have to replace this:

from LSTMDA_torch import LSTMDA, fit_torch_model, rmse_masked

with:

import importlib
LSTMDA = importlib.import_module('03b_model.src.LSTMDA_torch', 'LSTMDA')
fit_torch_model = importlib.import_module('03b_model.src.LSTMDA_torch', 'fit_torch_model')
rmse_masked = importlib.import_module('03b_model.src.LSTMDA_torch', 'rmse_masked')

This is because I am working out of the root directory \USGS-R\\drb-estuary-salinity-ml any time I'm running code. Are you only running this code in Snakemake? And is it working for you there?

I would have made a in-code suggestion, but I can't comment on that line since it wasn't in the diff for this particular pull request, so hopefully this comment in the main conversation thread is ok.

amsnyder commented 2 years ago

What was your process for importing river_dl.preproc_utils? Did you just clone the repo adjacent to drb-estuary-salinity-ml? Or do we want a stable copy of this file: https://github.com/USGS-R/river-dl/blob/main/river_dl/preproc_utils.py in our repo somewhere, in case they make changes?

galengorski commented 2 years ago

When I am importing LSTMDA_torch.py, I have to replace this:

from LSTMDA_torch import LSTMDA, fit_torch_model, rmse_masked

with:

import importlib
LSTMDA = importlib.import_module('03b_model.src.LSTMDA_torch', 'LSTMDA')
fit_torch_model = importlib.import_module('03b_model.src.LSTMDA_torch', 'fit_torch_model')
rmse_masked = importlib.import_module('03b_model.src.LSTMDA_torch', 'rmse_masked')

This is because I am working out of the root directory \USGS-R\\drb-estuary-salinity-ml any time I'm running code. Are you only running this code in Snakemake? And is it working for you there?

I would have made a in-code suggestion, but I can't comment on that line since it wasn't in the diff for this particular pull request, so hopefully this comment in the main conversation thread is ok.

I am running it through snakemake, maybe it will require a sys.path.append to point the python path to the right directory

galengorski commented 2 years ago

What was your process for importing river_dl.preproc_utils? Did you just clone the repo adjacent to drb-estuary-salinity-ml? Or do we want a stable copy of this file: https://github.com/USGS-R/river-dl/blob/main/river_dl/preproc_utils.py in our repo somewhere, in case they make changes?

I used git submodule init to initialize river-dl as a submodule in 03b_model/src/, it might be nice to have a stable copy in there, ahtough right now we are just using a few pre processing functions

amsnyder commented 2 years ago

What was your process for importing river_dl.preproc_utils? Did you just clone the repo adjacent to drb-estuary-salinity-ml? Or do we want a stable copy of this file: https://github.com/USGS-R/river-dl/blob/main/river_dl/preproc_utils.py in our repo somewhere, in case they make changes?

I used git submodule init to initialize river-dl as a submodule in 03b_model/src/, it might be nice to have a stable copy in there, ahtough right now we are just using a few pre processing functions

I'd like to add this step into the Snakefile if possible so it's totally reproducible - I haven't used git submodules before - did you run "git submodule add https://github.com/USGS-R/river-dl"? And which directory were you in when you ran it?