search

Riverscapes / NREI

0 stars 0 forks source link

Which scenarios do we want to run? #1

Open MattReimer opened 7 years ago

MattReimer commented 7 years ago

In order to automate this I need to know what the CSV file will be and how we will build it. i.e: how many combinations of parameters do we want to run per visit of NREI?

For Habitat we Code the simulations we want to run so we can version them. Each simulation corresponds to a species/lifestage combination.

like so:

    { id: 1, name: 'FIS - ch_jv', species: 'Chinook', lifestage: 'Juvenile', needD50: true },
    { id: 2, name: 'FIS - ch_sp', species: 'Chinook', lifestage: 'Spawner', needD50: true },
    { id: 3, name: 'FIS - st_jv', species: 'Steelhead', lifestage: 'Juvenile', needD50: true },
    { id: 4, name: 'FIS - st_sp', species: 'Steelhead', lifestage: 'Spawner', needD50: true },
    { id: 5, name: 'HSI - mr_ch_jv_gm', species: 'Chinook', lifestage: 'Juvenile', needD50: false },
    { id: 6, name: 'HSI - mr_ch_sp_gm', species: 'Chinook', lifestage: 'Spawner', needD50: true },
    { id: 7, name: 'HSI - mr_st_jv_gm', species: 'Steelhead', lifestage: 'Juvenile', needD50: false },
    { id: 8, name: 'HSI - mr_st_sp_gm', species: 'Steelhead', lifestage: 'Spawner', needD50: true },
  ],

So how's it going to work for NREI? For each visit how many runs of NREI are we expecting to do?

MattReimer commented 7 years ago
Matt Reimer (mattreimer)
Matt Reimer moved this card from Ongoing Discussions to Completed Discussions

Nov 24, 2016 at 2:22 PM

EW
Eric

Sounds good

Oct 4, 2016 at 11:45 AM - Reply - Delete

Matt Reimer (mattreimer)
Matt Reimer

@ewfhc ok. So I'm going to use this CSV File as my base for running NREI. At present it only has a single row but we can add more as necessary.

Oct 3, 2016 at 10:12 AM - Edit - Add Link as Attachment - Delete

EW
Eric

Hey guys. As far as making one of these .csv files by hand, could we just use the .csv I provded in the NREI example files as a starting point? My impression is that we could just add a column to this table for flow (and maybe remove some other variables (if @mattreimer has made them external to the input lookup table) and we would be set?

Sep 29, 2016 at 1:35 PM (edited) - Reply - Delete

EW
Eric

Just popped over to the AWS Automation Thoughts card and saw the message about deciding what fish/drift/temperature combos to run. Unless we want to make a change, those values are already hard-coded into the script (unless those were pulled out by @mattreimer to live external to the code). Regardless, it seems we can just use the values that were in there. Unless we are trying to change those? But I don't think we are.

Sep 29, 2016 at 9:22 AM - Reply - Add Link as Attachment - Delete

EW
Eric

Hey guys,

Apologies, I was out on holiday M-W.

Two thoughts: 1. What Pete said is spot-on. Basically, within one running of the code, it simulates ranges of fish sizes, temperatures, and drift values. So... you would want to do that for each visit-flow combo.

  1. @mattreimer, this may/may not be something that could be valuable from a time/cost perspective: It would be possible to limit outputting to the 'raw_outputs.zip' folder and then give users the ability to generate all of the other outputs from there. This is actually how I have been running the model recently. Doing this produces outputs from a single running of the code on the order of 10s-100s of MB, rather than GB+. This would significantly cut the data storage needs and the write-to-disk time (i.e. processor time). Functions for all of the outputs other than the NREI maps already exist. Just a thought.

Sep 29, 2016 at 9:15 AM - Reply - Delete

Matt Reimer (mattreimer)
Matt Reimer

@ewfhc I feel like the easiest way to do this is to do one by hand and we can look at it.

Would you be willing to give the combinations/permutations some thought and create a complete NREI run for one visit-flow? I don't think you need to run it, just let us see the initial CSV and R files you use with all the initial conditions.

Sep 28, 2016 at 1:27 PM - Edit - Delete

Matt Reimer (mattreimer)
Matt Reimer

@peteramchugh space is cheap(ish) but processing time is really what we care about. NREI takes a long time and uses a lot of disk space. The storage itself is less of a problem than the time it takes to generate and the network time to move it around.

I feel like we should start with the smallest set we can get away with and scale up.

Sep 28, 2016 at 1:24 PM - Edit - Delete

PM
Peter A Mchugh

I'm going to defer to EWall on this one, but my assumption is that for each visit-flow combo, results will be generated for a factorial combination of a range of fish sizes (size = species here), a range of temperatures, and a range of drift densities. Then, the output will be preserved for visit-flow-size-temp-drift combos for end users/consumers to harvest...

On an unrelated note, it sounds like we're going to do some drilling into the question of 'what drift should I use?' as an exercise led by Nick Weber. Maybe we can whittle the range down for some basins (unless space is free, then include/post-process at user stage). Something similar may then be in the works using Kris McNyset's temp model. Useful things to know, but not necessarily pertinent to this conversation.

Sep 28, 2016 at 10:31 AM - Reply - Delete

Matt Reimer (mattreimer)
Matt Reimer added Philip to this card

Sep 23, 2016 at 4:19 PM

Matt Reimer (mattreimer)
Matt Reimer added Peter A Mchugh to this card

Sep 23, 2016 at 4:19 PM

Matt Reimer (mattreimer)
Matt Reimer added Eric to this card

Sep 23, 2016 at 4:19 PM

Matt Reimer (mattreimer)
Matt Reimer added this card to Ongoing Discussions

Sep 23, 2016 at 4:17 PM