search

Riverscapes / riverscapes-tools

Open-source Python 3.0 tools for the Riverscapes organization
https://tools.riverscapes.net/
GNU General Public License v3.0
11 stars 9 forks source link

BRAT Validation Issues #42

Closed MattReimer closed 4 years ago

MattReimer commented 4 years ago

Based on a call we had with @mhallerud and @wally-mac

Possible unit problems in the equations

DRNAREA=sqMiles
ELEV=ft (5693)
FOREST=% (17.31)
PRECIP=in (21.77)

QLow
0.0247 * ((DRNAREA/0.3861021585424458)**1.05) * ((ELEV/3.28084)**-3.86) * (FOREST**-0.947) * ((PRECIP/0.0393701)**3.99)

Q2
0.00557 * ((DRNAREA/0.3861021585424458)**0.754) * ((PRECIP/0.0393701)**2.34)

here's an excerpt from the DB

PRECIP  PRECIP,PRECPRIS10   536.4481810897436
ELEV    ELEV    1735.325498590117
MINELEV MINELEV,MINBELEV    1334.19677734375
RR  RR  1707.66357421875
FOREST  FOREST  11.979218556596658
SLOP30_30M  SLOP30_30M  4.073632316368693
MEANSLOPE   MEANSLOPE   10.095673577064435

Here's the code that drives this: https://github.com/Riverscapes/sqlBRAT/blob/master/hydrology.py#L91-L96

Maggie sent some reference images.

Screen Shot 2020-05-21 at 1 25 03 PM Screen Shot 2020-05-21 at 1 25 23 PM Screen Shot 2020-05-21 at 1 25 41 PM Screen Shot 2020-05-21 at 1 25 59 PM
philipbaileynar commented 4 years ago

This is the equation that we are using for low flow in Station Creek (16010202):

0.0247 * ((DRNAREA/0.3861021585424458)**1.05) * ((ELEV/3.28084)**-3.86) * (FOREST**-0.947) * ((PRECIP/0.0393701)**3.99)

Earlier today we confirmed that the terms use the following units:

My ultimate question for @mhallerud is whether the value 0.3861021585424458 is from the cited USGS publication or whether this is an attempt to convert from drainage area values in our data stored in square kilometres to the necessary square miles for the equation to work?

1 square kilometre = 0.386102 square miles

So shouldn't this term be:

DRNAREA * 0.386102

I think it's a good idea if the equations used by the code are taken directly from the literature and that we keep all conversion factors elsewhere (I propose the database).

mhallerud commented 4 years ago

@philipbaileynar Not sure why there are extra digits, but yes the purpose of that value is converting from square kilometers to square miles.

philipbaileynar commented 4 years ago

@mhallerud was the conversion factor in the literature? Or was it added by ETAL to convert from our source data?

mhallerud commented 4 years ago

@philipbaileynar It was added by us, the literature just specified that drainage area was in square miles.

philipbaileynar commented 4 years ago

Hydrology

The problems with hydrology were caused by several unit conversion errors. I have simplified this greatly and am now getting comparable discharge and stream power values with pyBRAT.

high_stream_power

Vegetation

sqlBRAT was consistently under reporting the vegetation suitability. This was caused by an SQL query error that was double accounting the area of each buffer (because it was combining both existing and historical buffers during each query). So the proportion of any vegetation type was diminished because it was a fraction of twice the area. This is fixed.

existing_veg_suitability

Dam Capacity

The dam capacitty looks much more comparible. sqlBRAT has lower values in the central valley. This requires further investigation, but I am really pleased that the most egregious errors are fixed.

existing_dam_capacity

Shout out to @wally-mac @Cashe93 @joewheaton @MattReimer so they see this important update.

joewheaton commented 4 years ago

Really nice work all.... This is getting closer. At a cursory glance, almost everything in sqlBRAT that is red, which is orange in pyBRAT are canals. Any thoughts?

wally-mac commented 4 years ago

@philipbaileynar wow, this looks so much better! Nice work troubleshooting the hydrology and the veg. @philipbaileynar @joewheaton Do you guys have suggestions on how to further investigate the differences especially in the "central" portion of the watershed. Is this sqlBRAT run now available in the warehouse? @philipbaileynar @joewheaton If you guys think it's a good next step @Cashe93 let's interrogate these differences in outputs together and see if we can figure out what's driving the changes. Very impressive work. Big relief that we are making such great headway.

philipbaileynar commented 4 years ago

@wally-mac yes this latest version of middle bear is in the Anabranch warehouse.

I welcome any investigation that @Cashe93 can perform that sheds light.

I'm curious about @joewheaton "canal" comment and will investigate also.

Cashe93 commented 4 years ago

Sounds good @wally-mac. Nice work @philipbaileynar!

philipbaileynar commented 4 years ago

Here is a video demonstrating the new BRAT report that is produced with every sqlBRAT run. The goal is to make the inner workings and results of sqlBRAT more transparent. I know some folks are not yet comfortable with SQL databases, so this report gets important results into an easily accessible HTML format that anyone can view in a web browser to investigate a BRAT run.

https://youtu.be/aLsDlcYW8CM

The report code is unbelievably simple. We intend to extend it to include the electivity indices and other valuable summary information. We also hope to write similar reports for riverscapes context and VBET projects. And we encourage all model owners to consider doing the same for their models too.

wally-mac commented 4 years ago

@philipbaileynar Thank you for generating the BRAT report. I watched the video and found the report very useful. I agree, I think a report like this would be great to have for each of our tools. Thank you!

joewheaton commented 4 years ago

More importantly, what @philipbaileynar just made is easy enough that @tyler1218hatch or @Cashe93 could implement. Really powerful. It needs us to really think about exactly what we want to show and graph, but super cool. @kbartelt may like too.

philipbaileynar commented 4 years ago

Closing this issue now that the core task of getting sqlBRAT to produce comparable results to pyBRAT. Any subsequent validation issues should use a new issue. Refer back to this issue if relevant.