search

WikiWatershed / model-my-watershed

The web application front end for Model My Watershed.
https://modelmywatershed.org
Apache License 2.0
57 stars 31 forks source link

Update values for ET variable in Mapshed #2549

Closed ajrobbins closed 6 years ago

ajrobbins commented 6 years ago

From BME:

we are currently using an attribute field (Etadj) in our national weather station layer to determine the value for the "PctET(I)" variable that gets written to the last position of lines 8-19 in the gms file (this is the monthly ET adjustment factor for each station). After doing some initial model calibration checks, I have seen that the values we have been using in this field are not quite right for stations in the watersheds that I have been checking around the country. Consequently, I have revised these values, and have attached an updated version of the dbf file associated with our weather station shapefile. Would it be possible to replace the older values with these newer ones sometime in the near future?

[remove .txt] weathstaalb.dbf.txt

ajrobbins commented 6 years ago

Additional task:

When you replace the new ET values as described below, would it also be possible to make a small change in the equation used to calculate the value for the "SedNitr" factor in the model input (gms) file that is based on the "soil nitrogen" map used in MMW? Specifically, could we change the multiplication factor from "4.0" to "7.0" as shown in the attached document that has been updated from the previous version?

https://drive.google.com/open?id=0B0hs2mX4ApaLYnNkdThNZXppWUN2dE1USUp2X1hqZG9sekM0

ajrobbins commented 6 years ago

shares points with #2559

rajadain commented 6 years ago

The DBF file above contains five columns for ET Adjustment: ETADJ, ET_ADJ2, ET_ADJ3, ETADJ_ORIG, and ETADJ_NEW. Of these, ET_ADJ3 mirrors the current values, as shown here:

STATION LOCATION Current New
etadj ETADJ ET_ADJ2 ET_ADJ3 ETADJ_ORIG ETADJ_NEW
3103 Flagstaff 1.3 1.3 1.48 1.3 1.85 1.3
3812 Asheville 0.93 1.25 1.06 0.93 1.32 1.25
3813 Macon 0.96 1.25 1.09 0.96 1.36 1.25
3820 Augusta 0.96 1.25 1.09 0.96 1.36 1.25
3822 Savannah 0.92 1.25 1.04 0.92 1.3 1.25
3856 Huntsville 0.85 1.21 0.97 0.85 1.21 1.21
3860 Huntington 0.99 1.25 1.12 0.99 1.4 1.25
3870 Greenville/Spartanburg 0.89 1.27 1.02 0.89 1.27 1.27
3927 Fort Worth 1.09 1.4 1.24 1.09 1.55 1.4
3928 Wichita 1.16 1.32 1.32 1.16 1.65 1.32
3937 Lake Charles 0.87 1.23 0.98 0.87 1.23 1.23
3940 Jackson 0.87 1.23 0.98 0.87 1.23 1.23
3945 Columbia 1.01 1.16 1.15 1.01 1.44 1.16
3947 Kansa sCity 1.04 1.25 1.18 1.04 1.47 1.25
4725 Binghamton 1.04 1.25 1.18 1.04 1.48 1.25
4751 Bradford 0.95 1.16 1.08 0.95 1.35 1.16
12834 Daytona Beach 0.92 1.25 1.05 0.92 1.31 1.25
12836 Key West 1.01 1.25 1.14 1.01 1.43 1.25
12839 Miami 0.86 1.25 0.98 0.86 1.22 1.25
12842 Tampa 0.96 1.25 1.1 0.96 1.37 1.25
12844 West Palm Beach 0.83 1.25 0.94 0.83 1.18 1.25
12912 Victoria 1.04 1.2 1.18 1.04 1.47 1.2
12916 New Orleans 0.82 1.2 0.93 0.82 1.16 1.2
12917 Port Arthur 0.85 1.25 0.97 0.85 1.21 1.25
12919 Brownsville 1.21 1.72 1.38 1.21 1.72 1.72
12921 San Antonio 1.13 1.35 1.29 1.13 1.61 1.35
12924 Corpus Christi 1.15 1.63 1.3 1.15 1.63 1.63
12960 Houston 0.94 1.34 1.07 0.94 1.34 1.34
13722 Raleigh/Durham 0.99 1.25 1.12 0.99 1.4 1.25
13723 Greensboro 0.98 1.16 1.11 0.98 1.39 1.16
13729 Elkins 0.95 1.16 1.08 0.95 1.35 1.16
13733 Lynchburg 0.99 1.3 1.13 0.99 1.41 1.3
13737 Norfolk 0.96 1.25 1.09 0.96 1.36 1.25
13739 Philadelphia 0.99 1.25 1.12 0.99 1.4 1.25
13740 Richmond 0.97 1.16 1.1 0.97 1.38 1.16
13741 Roanoke 0.99 1.25 1.13 0.99 1.41 1.25
13748 Wilmington 0.87 1.16 0.99 0.87 1.24 1.16
13781 Wilmington 0.99 1.25 1.13 0.99 1.41 1.25
13865 Meridian 0.85 1.21 0.97 0.85 1.21 1.21
13866 Charleston 0.98 1.16 1.11 0.98 1.39 1.16
13873 Athens 0.91 1.25 1.03 0.91 1.29 1.25
13874 Atlanta 0.9 1.28 1.02 0.9 1.28 1.28
13876 Birmingham 0.87 1.24 0.99 0.87 1.24 1.24
13877 Bristol 1 1.25 1.14 1 1.42 1.25
13880 Charleston 0.89 1.25 1.02 0.89 1.27 1.25
13881 Charlotte 0.97 1.16 1.1 0.97 1.38 1.16
13882 Chattanooga 0.88 1.16 1 0.88 1.25 1.16
13883 Columbia 0.91 1.16 1.03 0.91 1.29 1.16
13889 Jacksonville 0.89 1.27 1.02 0.89 1.27 1.27
13891 Knoxville 0.93 1.25 1.06 0.93 1.32 1.25
13893 Memphis 0.89 1.16 1.01 0.89 1.26 1.16
13894 Mobile 0.81 1.15 0.92 0.81 1.15 1.15
13895 Montgomery 0.88 1.16 1 0.88 1.25 1.16
13897 Nashville 0.93 1.25 1.06 0.93 1.32 1.25
13957 Shreveport 0.94 1.34 1.07 0.94 1.34 1.34
13958 Austin 1.12 1.59 1.27 1.12 1.59 1.59
13959 Waco 1.11 2.2 1.26 1.11 1.58 2.2
13962 Abilene 1.27 1.5 1.44 1.27 1.8 1.5
13963 Little Rock 0.9 1.28 1.02 0.9 1.28 1.28
13964 Fort Smith 0.99 1.3 1.13 0.99 1.41 1.3
13966 Wichita Falls 1.17 2 1.33 1.17 1.66 2
13967 Oklahoma City 1.09 1.6 1.24 1.09 1.55 1.6
13968 Tulsa 1 1.6 1.14 1 1.42 1.6
13970 Baton Rouge 0.82 1.3 0.94 0.82 1.17 1.3
13985 Dodge City 1.34 1.9 1.52 1.34 1.9 1.9
13994 St Louis 1.04 1.25 1.18 1.04 1.47 1.25
13995 Springfield 0.97 1.16 1.1 0.97 1.38 1.16
13996 Topeka 1.06 1.25 1.21 1.06 1.51 1.25
14607 Caribou 1.05 1.4 1.19 1.05 1.49 1.4
14733 Buffalo 1.02 1.25 1.16 1.02 1.45 1.25
14734 Newark 0.96 1.25 1.1 0.96 1.37 1.25
14735 Albany 1.06 1.25 1.2 1.06 1.5 1.25
14737 Allentown 0.96 1.16 1.1 0.96 1.37 1.16
14739 Boston 0.99 1.4 1.12 0.99 1.4 1.4
14740 Hartford 0.96 1.25 1.09 0.96 1.36 1.25
14742 Burlington 1.08 1.5 1.22 1.08 1.53 1.5
14745 Concord 1.05 1.4 1.19 1.05 1.49 1.4
14751 Harrisburg 1 1.16 1.14 1 1.42 1.16
14764 Portland 0.96 1.36 1.09 0.96 1.36 1.36
14765 Providence 0.94 1.34 1.07 0.94 1.34 1.34
14768 Rochester 1.11 1.25 1.26 1.11 1.58 1.25
14771 Syracuse 1.02 1.25 1.16 1.02 1.45 1.25
14777 WB/Scranton 1.05 1.25 1.19 1.05 1.49 1.25
14778 Williamsport 1 1.16 1.14 1 1.42 1.16
14820 Cleveland 1.05 1.16 1.19 1.05 1.49 1.16
14821 Columbus 1.03 1.25 1.17 1.03 1.46 1.25
14826 Flint 1.14 1.14 1.3 1.14 1.62 1.14
14827 Fort Wayne 1.07 1.2 1.22 1.07 1.52 1.2
14836 Lansing 1.14 1.27 1.3 1.14 1.62 1.27
14837 Madison 1.13 1.13 1.29 1.13 1.61 1.13
14839 Milwaukee 1.1 1.2 1.25 1.1 1.56 1.2
14840 Muskegon 1.11 1.25 1.26 1.11 1.57 1.25
14842 Peoria 1.05 1.2 1.19 1.05 1.49 1.2
14847 Sault Ste Marie 1.08 1.25 1.22 1.08 1.53 1.25
14848 South Bend 1.01 1.16 1.15 1.01 1.44 1.16
14850 Traverse City 1.15 1.27 1.31 1.15 1.64 1.27
14852 Youngstown 1.04 1.2 1.18 1.04 1.47 1.2
14860 Erie 0.99 1.25 1.12 0.99 1.4 1.25
14891 Mansfield 1.01 1.16 1.14 1.01 1.43 1.16
14895 Akron/Canton 1.04 1.16 1.18 1.04 1.48 1.16
14898 Bay 1.17 1.17 1.33 1.17 1.66 1.17
14913 Duluth 1.15 1.27 1.3 1.15 1.63 1.27
14914 Fargo 1.41 1.41 1.6 1.41 2 1.41
14918 International Falls 1.27 1.18 1.44 1.27 1.8 1.18
14920 La Crosse 1.14 1.14 1.3 1.14 1.62 1.14
14922 Minneapolis/St Paul 1.18 1.27 1.34 1.18 1.68 1.27
14923 Moline 1.01 1.16 1.15 1.01 1.44 1.16
14925 Rochester 1.15 1.27 1.31 1.15 1.64 1.27
14926 SaintCloud 1.2 1.27 1.36 1.2 1.7 1.27
14933 Des Moines 1.1 1.25 1.25 1.1 1.56 1.25
14935 Grand Island 1.25 1.25 1.42 1.25 1.78 1.25
14936 Huron 1.39 1.39 1.58 1.39 1.97 1.39
14940 Mason City 1.11 1.25 1.26 1.11 1.57 1.25
14941 Norfolk 1.25 1.25 1.42 1.25 1.77 1.25
14943 Sioux City 1.23 1.4 1.4 1.23 1.75 1.4
14944 SiouxFalls 1.27 1.45 1.45 1.27 1.81 1.45
14991 Eau Claire 1.12 1.12 1.27 1.12 1.59 1.12
23023 Midland/Odessa 1.58 1.3 1.8 1.58 2.25 1.3
23034 San Angelo 1.37 1.37 1.56 1.37 1.95 1.37
23042 Lubbock 1.43 1.62 1.62 1.43 2.03 1.62
23044 El Paso 2.03 2.3 2.3 2.03 2.88 2.3
23047 Amarillo 1.4 1.99 1.59 1.4 1.99 1.99
23048 Tucumcari 1.57 2.23 1.78 1.57 2.23 2.23
23050 Albuquerque 2.02 2.3 2.3 2.02 2.87 2.3
23061 Alamosa 2.18 2.18 2.48 2.18 3.1 2.18
23063 Eagle 1.86 1.86 2.11 1.86 2.64 1.86
23065 Goodland 1.45 1.65 1.65 1.45 2.06 1.65
23066 Grand Junction 2.05 2.05 2.33 2.05 2.91 2.05
23129 Long Beach 1.77 1.77 2.02 1.77 2.52 1.77
23153 Tonopah 2.49 2.49 2.82 2.49 3.53 2.49
23154 Ely 1.9 2.16 2.16 1.9 2.7 2.16
23155 Bakersfield 2.49 2.49 2.82 2.49 3.53 2.49
23160 Tucson 1.76 1.76 2 1.76 2.5 1.76
23161 Daggett 2.95 3.35 3.35 2.95 4.19 3.35
23169 Las Vegas 2.89 3.28 3.28 2.89 4.1 3.28
23174 Los Angeles 1.76 1.77 2 1.76 2.5 1.77
23183 Phoenix 2.16 2.46 2.46 2.16 3.07 2.46
23185 Reno 2.18 2.48 2.48 2.18 3.1 2.48
23188 San Diego 1.92 1.92 2.18 1.92 2.73 1.92
23232 Sacramento 1.47 1.67 1.67 1.47 2.09 1.67
23234 SanFrancisco 1.39 1.39 1.58 1.39 1.98 1.39
23273 SantaMaria 1.73 1.73 1.97 1.73 2.46 1.73
24011 Bismarck 1.56 1.56 1.78 1.56 2.22 1.56
24013 Minot 1.46 1.46 1.66 1.46 2.08 1.46
24018 Cheyenne 1.61 1.61 1.83 1.61 2.29 1.61
24021 Lander 1.7 1.7 1.93 1.7 2.41 1.7
24023 NorthPlatte 1.41 1.6 1.6 1.41 2 1.6
24025 Pierre 2.03 2.03 2.3 2.03 2.88 2.03
24027 Rock Springs 1.96 1.85 2.23 1.96 2.79 1.85
24028 Scottsbluff 1.57 1.57 1.78 1.57 2.23 1.57
24029 Sheridan 1.61 1.61 1.83 1.61 2.29 1.61
24033 Billings 1.58 1.58 1.8 1.58 2.25 1.58
24036 Lewistown 1.27 1.44 1.44 1.27 1.8 1.44
24037 Miles City 1.63 1.63 1.86 1.63 2.32 1.63
24089 Casper 1.73 1.73 1.96 1.73 2.45 1.73
24090 RapidCity 1.51 1.71 1.71 1.51 2.14 1.71
24121 Elko 1.92 2.18 2.18 1.92 2.73 2.18
24127 Salt Lake City 1.53 1.74 1.74 1.53 2.17 1.74
24128 Winnemucca 2.1 2.38 2.38 2.1 2.98 2.38
24131 Boise 1.75 1.75 1.99 1.75 2.49 1.75
24137 Cut Bank 1.77 2.02 2.02 1.77 2.52 2.02
24143 Great Falls 1.58 1.79 1.79 1.58 2.24 1.79
24144 Helena 1.79 1.79 2.03 1.79 2.54 1.79
24146 Kalispell 1.51 1.51 1.72 1.51 2.15 1.51
24153 Missoula 1.67 1.67 1.9 1.67 2.37 1.67
24155 Pendleton 1.76 2 2 1.76 2.5 2
24156 Pocatello 1.75 1.75 1.99 1.75 2.49 1.75
24157 Spokane 1.51 1.51 1.72 1.51 2.15 1.51
24221 Eugene 0.91 1.16 1.03 0.91 1.29 1.16
24225 Medford 1.42 1.62 1.62 1.42 2.02 1.62
24227 Olympia 0.9 1 1.02 0.9 1.28 1
24229 Portland 1.05 1.25 1.19 1.05 1.49 1.25
24230 Redmond/Bend 2.06 2.06 2.34 2.06 2.92 2.06
24232 Salem 1.01 1.16 1.15 1.01 1.44 1.16
24233 Seattle/Tacoma 1.04 1.04 1.18 1.04 1.48 1.04
24243 Yakima 2.13 1.8 2.42 2.13 3.02 1.8
24283 Arcata 1.04 1.18 1.18 1.04 1.47 1.18
24284 North Bend 0.81 1.15 0.92 0.81 1.15 1.15
93037 Colorado Springs 1.53 1.53 1.74 1.53 2.17 1.53
93058 Pueblo 1.82 1.82 2.06 1.82 2.58 1.82
93129 Cedar City 1.8 1.8 2.04 1.8 2.55 1.8
93193 Fresno 1.87 1.87 2.12 1.87 2.65 1.87
93721 Baltimore 0.99 1.25 1.13 0.99 1.41 1.25
93729 Cape Hatteras 0.86 1.16 0.98 0.86 1.22 1.16
93730 Atlantic City 1 1.16 1.14 1 1.42 1.16
93738 Sterling 1 1.2 1.14 1 1.42 1.2
93805 Tallahassee 0.8 1.13 0.9 0.8 1.13 1.13
93814 Covington 0.99 1.25 1.13 0.99 1.41 1.25
93815 Dayton 1.05 1.16 1.19 1.05 1.49 1.16
93817 Evansville 0.97 1.16 1.1 0.97 1.38 1.16
93819 Indianapolis 1.01 1.16 1.14 1.01 1.43 1.16
93820 Lexington 0.96 1.16 1.09 0.96 1.36 1.16
93821 Louisville 0.96 1.16 1.09 0.96 1.36 1.16
93822 Springfield 1.06 1.25 1.21 1.06 1.51 1.25
93842 Columbus 0.89 1.16 1.02 0.89 1.27 1.16
93987 Lufkin 0.95 1.16 1.08 0.95 1.35 1.16
94008 Glasgow 1.84 2.09 2.09 1.84 2.61 2.09
94018 Boulder 1.56 1.56 1.78 1.56 2.22 1.56
94224 Astoria 0.8 0.91 0.9 0.8 1.13 0.91
94240 Quillayute 0.64 0.91 0.73 0.64 0.91 0.91
94702 Bridgeport 0.99 1.25 1.12 0.99 1.4 1.25
94725 Massena 1.08 1.5 1.23 1.08 1.54 1.5
94728 New York City 0.93 1.16 1.06 0.93 1.32 1.16
94746 Worcester 0.92 1.54 1.05 0.92 1.31 1.54
94814 Houghton Lake 1.18 1.27 1.34 1.18 1.68 1.27
94822 Rockford 1.05 1.25 1.19 1.05 1.49 1.25
94823 Pittsburgh 1.04 1.16 1.18 1.04 1.48 1.16
94830 Toledo 1.1 1.2 1.25 1.1 1.56 1.2
94846 Chicago AP 1.06 1.1 1.2 1.06 1.5 1.1
94847 Detroit Metro AP 1.11 1.25 1.26 1.11 1.57 1.25
94849 Alpena 1.17 1.25 1.33 1.17 1.66 1.25
94860 Grand Rapids 1.27 1.27 1.44 1.27 1.8 1.27
94910 Waterloo 1.09 1.25 1.24 1.09 1.55 1.25
94918 Omaha 1.16 1.32 1.32 1.16 1.65 1.32

Which column should be used to update the values with? ETADJ_NEW?

ajrobbins commented 6 years ago

From Barry:

You use the same variable as you have been using in the code (ETADJ). The other fields are just placeholders for me doing different calculations. I have already set the older ET values to my preferred ones in the ETADJ field.

rajadain commented 6 years ago

Also, it seems like our current ms_weather_station.sql data file has all the rows twice, identically. Not sure why that is, but as part of this update I'm going to make sure there's only one of each.

rajadain commented 6 years ago

I've updated ms_weather_stations with the new ET values, and also updated the SedNitr factor to 7.0 from 4.0. The difference for Little Neshaminy (production with old values left, local with new values right) are:

image

As can be seen, the PrcntET(i) values in lines 8-19 position 8 are updated with the new ET values, and SedNitr in line 37 position 1 is updated with the new factor.

For some reason I'm also seeing differences in weather data on my local, that isn't observed on Staging or Production. I'm trying to investigate the root cause of that and ensure that the changes in this PR do not make an impact there.

rajadain commented 6 years ago

After a long day of investigation, I've found the root cause of the change in weather data: previously we mistakenly had 2 of each entry in the ms_weather_station table. When we would select the top 2 matching weather stations for any area of interest, we would always get 2 identical stations. I removed the extraneous duplication in the updated script, and now we select 2 actually different stations. This causes new values for weather data. This also makes #2561 more urgent.

However, this change is successful and can be completed. Opening a PR now.