search

vicelab / cen-sierra-pywr

Code base for modeling the central Sierra Nevada hydropower systems
3 stars 5 forks source link

Notes on overall model performance #138

Open GustavoFacincani opened 3 years ago

GustavoFacincani commented 3 years ago

Below there are table with comments on model performance of each facility (i.e., storage, instreamflow requirements (IFRs), hydropower generation, hydropower flow and outflows) in each basin. These notes are based on the comparison of model results to observed data. Average daily values per month were used for comparing the observed data measured by USGS gauges in each location (1980-2011) for instreamflows, hydropower flows, storage and outflows, meanwhile the electricity generation was based on monthly totals, as the observed data provided by the Energy Information Administration (EIA). As the basin models are based upon the information provided by the licenses and their updates, certain patterns of operations might differ overtime (due to previous updates) or are not captured due to different operations determined by the operator's decisions or limitations in the hydrology (Livneh data). In the observed data, it is noticeable some facilities presented different patterns of operations at certain times in the past. For instance, some hydropower flows showed a different pattern of operation prior to 1994/1995, certain IFRs showed changes in flow regimes overtime or had flows higher than the minimum requirements delivered downstream. These differences affect the model performance metrics (NSE, PBIAS and RSR/RMSE).

Instreamflow Requirements Basin Instream Flow Requirement Comments
stanislaus Donnell Lake Spill No data
  IFR at Murphys Park Ok
  IFR bl Angels Div Only about 10 years of data are available, in which the first 3-4 years the IFR followed a different pattern, some peaks are only present in the modeled data
  IFR bl Beardsley Afterbay Ok, IFRs were higher before 1986, peak flows are caught but greater in the modeled data
  IFR bl Beaver Creek Diversion Dam Good, few peaks are missed
  IFR bl Collierville PH discharge Behavior is good, but the modeled data has many peak flows that are only present in the modeled data
  IFR bl confluence of NF Stanislaus and Beaver Creek Good, few peaks are missed
  IFR bl Donnell Lake Only ~5 years of data, first ~1.5 don't match well
  IFR bl Goodwin Reservoir Good, although some of the small/intermediate peaks are greater in the modeled data
  IFR bl Hunter Reservoir Bad/weird/mostly missing observed data
  IFR bl Lyons Res Good
  IFR bl McKays Point Div Good
  IFR bl New Spicer Meadow Reservoir IFRs match better before 1990, after that the lower flows tend to be higher and vary more
  IFR bl NF Stanislaus Div Res Minimum flows are higher in the observed, meanwhile the modeled data has many peaks, none present in the observed
  IFR bl Philadelphia Div Good, although data is sparse over the years, present mostly when lower flows occur in the modeled data, when peak flows occur in the modeled data, the observed is blank (bad data)
  IFR bl Pinecrest Lake No data
  IFR bl Relief Reservoir No data
  IFR bl Sand Bar Div Sparse data, mostly missing; when present, there are no peak flows, meanwhile the modeled data has many
  IFR bl Utica Reservoir Few consecutive months of observations during 5 years, all with low flows, meanwhile peak flows are occurring in the modeled data
  New Melones Lake Flood Control No data
tuolumne Don Pedro Lake Spillway No data
  IFR at La Grange Good, a few small peaks missed
  IFR bl Cherry Lake Good, a few small peaks missed
  IFR bl Hetch Hetchy Reservoir Good, a few small peaks missed
  IFR bl Lake Eleanor Good, a few small peaks missed
  Moccasin Fish Hatchery No data
upper_san_joaquin IFR above Shakeflat Creek Good, although overproducing
  IFR bl Balsam Forebay Bad stats due to a few peak flows not present in the observed
  IFR bl Bass Lake Good, although some peak flows were missed
  IFR bl Bear Div Good
  IFR bl Big Creek 5 Div Some high flows are missed
  IFR bl Big Creek 6 Div Good
  IFR bl Bolsillo Creek Div Bad/weird/mostly missing observed data
  IFR bl Browns Creek Ditch No data
  IFR bl Camp 62 Creek Div Bad/weird/mostly missing observed data
  IFR bl Chinquapin Creek Div Bad/weird/mostly missing observed data
  IFR bl Hooper Creek Good, although some peak flows were missed
  IFR bl Huntington Lake MIFs started to be followed only recently (2010-2011, which are the latest observations used)
  IFR bl Kerckhoff Lake Good
  IFR bl Lake Thomas A Edison Some smaller high flows are missed meanwhile higher high flows are only present in the modeled data
  IFR bl Manzanita Div No data
  IFR bl Millerton Lake Rules constantly changing
  IFR bl Mono Creek Div Good, some peak flows missed in the modeled data
  IFR bl Pitman Creek Div Ok, some peak flows missed in the modeled data
  IFR bl Redinger Lake Good
  IFR bl San Joaquin 1 Div No data
  IFR bl San Joaquin R and Willow Cr confluence No data
  IFR bl Shaver Lake Some peaks flows missed, the ones captured are higher in the modeled data
  IFR No. Fk. Stevenson Creek above Shaver Lake Peak flows occur with a certain frequency in the observed, but not in the modeled data

Hydropower Generation

Basin Facility Comments
Stanislaus Angels PH Behavior matches better the newest data, although the modeled data has higher maximum generation
  Beardsley PH Good, although some peaks are higher in the modeled data
  Collierville PH Good, although the lower generation periods in the observed are not as low as in the modeled data
  Donnells PH Ok, although the lower generation periods in the observed are not as low as in the modeled data, mainly after 2000
  Murphys PH Generation tends to be twice as high in the observed, in many times
  New Melones PH Good, few peak generation events missed in the modeled data
  New Spicer Meadow PH Only about 4 years of data spread in 3 different times
  Phoenix PH Generation tends to be much higher in the observed
  Sand Bar PH Behavior matches better after 1995, although data is only available for ~1986-2001
  Spring Gap PH Timing doesn't match well, and modeled generation is a little bit higher than observed
  Stanislaus PH Modeled generation is a little bit higher than observed, timing doesn't match very well
Tuolumne Dion R Holm PH Observed has a few period with lower generation that are missed
  Don Pedro PH Good, although I lowered the capacity to avoid overgeneration, so it misses a few period of higher generation
  Kirkwood PH Behavior matches well after 1995
  Moccasin PH Behavior matches well after 1995, and there's a weird under generation followed by a very high overgeneration in the observed in 1991
Merced McSwain PH Overall good, besides a very high peak generation occurred in the observed in 2007
  Merced Falls PH Overall good, although the observed data has more variation (periods of lower and higher generation)
  New Exchequer PH Overall good, besides a very high peak generation occurred in the observed in 2007
Upper San Joaquin Big Creek 1 PH Modeled data has some higher peaks more recently; Behavior doesn't match very well mostly before 1994
  Big Creek 2 PH Behavior doesn't match well mostly before 1994
  Big Creek 2A PH Behavior doesn't match well mostly before 1995
  Big Creek 3 PH Good
  Big Creek 4 PH Good
  Big Creek 8 PH Modeled data has some higher peaks more recently; Behavior doesn't match well mostly before 1994
  Eastwood PH Overall good, besides a few higher peaks missed, compared to observed
  Friant PH Weird/bad observed data
  Kerckhoff 1 PH Most unmatches are in the lower generation periods; Behavior doesn't match well mostly before 1994
  Kerckhoff 2 PH Good
  Mammoth Pool PH Good (although the flow capacity was reduced to restrict overflows)
  Portal PH Observed generation was higher than modeled in the past, but declined after 2004
  San Joaquin 1 PH Observed has more periods with lower generation
  San Joaquin 1A PH Capacity is greater in the observed, but it has more periods with lower generation than higher than modeled
  San Joaquin 2 PH Behavior doesn't match well mainly before 1994
  San Joaquin 3 PH Observed has more variation (periods with lower and greater generation unmatched)

Hydropower Flow

Basin Facility Comments
Stanislaus Angels PH No data
  Beardsley PH Timing doesn't match very well, in few events, observed has a little bit higher flows
  Collierville PH Good, although the lower flow periods in the observed are not as low as in the modeled data
  Donnells PH No data
  Murphys PH Ok
  New Melones PH Good, few peak flow events missed
  New Spicer Meadow PH Lower flows tend to be higher and vary more
  Phoenix PH No data
  Sand Bar PH Behavior is good, but flows vary a lot (many lower and higher flows are missed)
  Spring Gap PH Timing doesn't match well
  Stanislaus PH Modeled flows are a bit higher than observed, timing doesn't match very well
Tuolumne Dion R Holm PH Ok, modeled data matches observed better after 1991
  Don Pedro PH No data
  Kirkwood PH Good
  Moccasin PH Data available only for ~1991-2002, it matches better after 1995
Upper San Joaquin Big Creek 1 PH Behavior doesn't match well, mostly before 1994
  Big Creek 2 PH Behavior doesn't match well, mostly before 1994
  Big Creek 2A PH Timing of flows don't match well
  Big Creek 3 PH Good
  Big Creek 4 PH Good
  Big Creek 8 PH Good overall
  Eastwood PH Good overall, although lower flows tend to be higher
  Friant PH No data
  Kerckhoff 1 PH Ok, modeled data missed some of the lower peaks
  Kerckhoff 2 PH Good
  Mammoth Pool PH Good (although the flow capacity was reduced to restrict overflows)
  Portal PH Ok
  San Joaquin 1 PH Timing of flows don't match well
  San Joaquin 1A PH Observed has greater variation, and greater flow capacity
  San Joaquin 2 PH Timing of flows don't match well
  San Joaquin 3 PH Timing of flows don't match well

Storage

Basin Facility Comments
Stanislaus Beardsley Reservoir Good
  Donnells Reservoir Good
  Hunter Reservoir No data
  Lake Tulloch Good, besides 7 random drawdowns in water levels in the observed
  Lyons Reservoir Behavior is good, but flows vary a lot in the observed (many lower and higher flows are missed)
  New Melones Lake Good, although water levels in ~1981-1983 and ~1989-1995 dropped much more in the observed
  New Spicer Meadow Reservoir The observed data has much more variation, in which lake levels drop more than 2/3, meanwhile it's mostly full in the modeled data (probably related to the agricultural deliveries that are not added to the model, as there is no data for it)
  Pinecrest Reservoir Overall good, although at times the reservoir empties less in the observed
  Relief Reservoir Overall good, although at times the reservoir empties more in the observed
  Union-Utica Reservoir Good, but only about 5 years of data are available, spread in 6 different times
Tuolumne Cherry Lake Ok, periods unmatched are due to greater or lower declines in water levels in the observed
  Don Pedro Reservoir Good
  Hetch Hetchy Reservoir Good, most unmatched periods are due to greater declines in water levels in the observed
  Lake Eleanor Ok, most unmatched periods are due to greater declines in water levels in the observed
Merced Lake McClure Good
  Lake McSwain Just one month of data available
Upper San Joaquin Bass Lake It is lacking water in some years, causing the levels to drop and keep low, more than in the observed
  Florence Lake Good overall
  Huntington Lake Levels usually drop more/for a longer time in the modeled data between Aug-Apr
  Kerckhoff Lake Besides 3 deeper drops, lake levels mostly vary in the same range of the modeled data, although they don't match well, the range of variation is little
  Lake Thomas A Edison Behavior is good, some drops and rises are more accentuated in the modeled data
  Mammoth Pool Reservoir Good overall, some drops are a little lower than in the observed
  Millerton Lake In many years the lake levels are low in the modeled data
  Redinger Lake There are many more and deeper drops in lake levels in the modeled data
  Shaver Lake Some years had lower lake levels in the observed (mostly before 1994)

Outflows

Basin Location Comments
stanislaus Oakdale Irrigation District Good
  Phoenix Canal Outflow No data
  South San Joaquin Irrigation District Good
  Stanislaus River Outflow Good, although some of the small/intermediate peaks are greater in the modeled data
tuolumne Groveland No data
  Modesto Irrigation District Good
  SFPUC Observed has many periods with lower demands (many accentuated drops in deliveries)
  Tuolumne River Outflow Good
  Turlock Irrigation District Good
merced Merced River Outflow Good
  MID Main Good
  MID Northside Good
upper_san_joaquin CVP Friant-Kern Canal Good
  CVP Madera Canal Releases are, at times, higher or lower than in the observed, but behavior is good
  San Joaquin River Outflow IFR is higher than most of the observed (after 2010, the minimum flows become higher), although the observed had some greater peak flows