Infiltration Editor Meeting

[FLO-2DKaren]

@rpachaly I figured out a great project that will engage your engineering skills and your python skills. Let's do a meeting and see if we can start a plan.

[FLO-2DKaren]

@rpachaly

Thanks, I got the Alabama Data.

I'm referring to this Maricopa data. I'm not using the other data. This is the Brand New data from this year.

https://flo-2d.sharefile.com/d-s843f67b0b5644341aa5a0ad258d295db

[rpachaly]

Yeah, I've a project with this data. In this project, the CRS units are in feet (in/hr) and the GAParametersFinal_June2023_rep XKSAT2006 are in mm/hr. I converted the data and on the soil_layer we have hydc and hydc_si.

maricopa.zip

I also ran some statistics here:

soil_layer {'CV': 0.6680307651895053, 'FIRSTQUARTILE': 0.076, 'IQR': 0.178, 'MAJORITY': 0.254, 'MAX': 0.762, 'MEAN': 0.156027397260274, 'MEDIAN': 0.152, 'MIN': 0.025, 'MINORITY': 0.762, 'RANGE': 0.737, 'STD_DEV': 0.10423110158230778, 'THIRDQUARTILE': 0.254}

GAParametersFinal_June2023_rep {'CV': 0.9450196695093711, 'FIRSTQUARTILE': 0.049, 'IQR': 0.199, 'MAJORITY': 0.248, 'MAX': 0.99, 'MEAN': 0.14055045871559624, 'MEDIAN': 0.093, 'MIN': 0.022, 'MINORITY': 0.156, 'RANGE': 0.968, 'STD_DEV': 0.13282294804480327, 'THIRDQUARTILE': 0.248}

[rpachaly]

@FLO-2DKaren I'm facing a tropical thunderstorm here and my connection started to get really unstable. I'll have to call it a day. Tomorrow I'll be back working on this.

[FLO-2DKaren]

@rpachaly OK sounds good.

[FLO-2DKaren]

@rpachaly When you get back. I'm looking in the Chapter 4 document and it is also showing Sandy Loam to be closer to 0.3 to 0.5.

[FLO-2DKaren]

I don't think that data is in mm/hr. It should also be in in/hr.

[rpachaly]

Hi Karen,

You were right, the data is in in/hr. I was checking the code and there are some import points to be made. My code now follows exactly the spreadsheet available at https://hrsl.ba.ars.usda.gov/SPAW/Index.htm, which is the spreadsheet used to check the computations made using the NDOT manual. The code and the spreadsheet results are matching. Here is the spreadsheet: Soil Water Characteristics-Equations.xls

1. On the NDOT manual, when the XKSAT is calculated, there is a constant (0.0393700787) being applied to the calculating that is not present in the spreadsheet. This is probably a unit conversion. I removed because I my calculations are performed in SI units and then transformed to the user's unit.
2. The min and maximum values on the NDOT manual are in in/hr. My code was setting these values in mm/hr. Now it is fixed. XKSAT min is 0.254 mm/hr (0.01 in/hr) or 50.8 mm/hr (2 in/hr).
3. The muname is added to the soil_layer.

I used the maricopa data to check the results. This is the project that I'm using: maricopa_2.zip. The results are very close now and, in most features, they are matching.

If you think that we are achieving good results, I can push the branch.

[FLO-2DKaren]

@rpachaly Yes for sure push the branch so I can see your calculations too.

This is great news.

[rpachaly]

I pushed the branch!

[rpachaly]

I've been researching about the best way to achieve our goal on the land use.

QuickOSM (https://plugins.qgis.org/plugins/QuickOSM/) Pros: Download directly the shapefiles. We can filter out many undesired data by creating very specific queries. Cons: It does not download correctly all the required shapes. Sometimes we have the data on OSM but QuickOSM does not download it. This will cause some empty spaces on the land use layer. We also have the problem with the roads (lines).

Overpass API (https://towardsdatascience.com/loading-data-from-openstreetmap-with-python-and-the-overpass-api-513882a27fd0) Pros: We can create the queries as we wish. There is no constrains on how to get the data. Cons: We'll have to code all the queries by hand. This overpass API is the same that QuickOSM uses. So, I think we'll have the same cons as the QuickOSM

Google Earth Engine QGIS plugin (https://gee-community.github.io/qgis-earthengine-plugin/) Pros: We'll have access to many datasets, including this one https://gee-community-catalog.org/projects/urban-watch/ Cons: The user needs to have an active Google Earth Engine (EE) account to use the plugin.

QuickMapServices: Pros: We can export the OSM Standard map to raster and then classify based on the OSM map legend (https://www.openstreetmap.org/key). Using this approach, there will be no empty areas and we can use the NDOT tables to estimate the IA, RTIMP and VC. The roads will be polygons. Cons: We will have to perform many processes (such as downloading the OSM map, clipping, raster calculator, vector clip/join/merge). Depending on the resolution and computational domain, this will take a long time to perform. Also, there is the labels that could have some impact in defining the land use areas.

In my opinion, none of these approaches is perfect. But the one that will give us less errors is the last option. I think it is going to be a slow process but good results will be achieved.

[FLO-2DKaren]

@rpachaly

What if we don't need to apply standard processors because we use the raster data instead of polygon data for the land use. This complies with the last option pretty well. We don't need that data to be in polygon format. It still means processing but I think we could do it with zonal statistics or something like that.

[rpachaly]

That's a good approach. It will make everything faster.

[rpachaly]

@FLO-2DKaren Hi Karen,

I did everything by hand today to see if it is doable. Here are the results:

I liked the results. It is a waaaay better than using the NLCD for urban areas. We can select everything that we want from the OSM based on the color. Then we can use the NDOT table to estimate IA, VC, and RTIMP.

I'll start coding.

[FLO-2DKaren]

Yes that looks really good.

[rpachaly]

@FLO-2DKaren Hi Karen,

I'm finishing up the land use tool, just need to run some tests here. Everything is looking really good. Can we have a meeting this afternoon?

[FLO-2DKaren]

@rpachaly Hi Robson, That's awsome!!! I can do a meeting.

Is Noon NY time OK? I'm working on something cool too. I'm reading timdep.hdf5 for the first time any cell > 0.01 depth and then writing that back to a new hdf5 dataset.

[rpachaly]

That works for me!

Awesome! Are you using compression on the hdf5 file?

[FLO-2DKaren]

@rpachaly Here is a meeting link. I'm not using compression. Noemi used compression on her hdf5 files and I was able to test them successfully.

GreenAmpt Meeting Jul 19, 2023, 12:00 – 12:30 PM (America/New_York)

Please join my meeting from your computer, tablet or smartphone. https://meet.goto.com/311587053 You can also dial in using your phone. Access Code: 311-587-053 United States: +1 (408) 650-3123

• One-touch: tel:+14086503123,,311587053#

Get the app now and be ready when your first meeting starts: https://meet.goto.com/install

[FLO-2DKaren]

@rpachaly Robson, Don't join the GoToMeeting.

Use this one To join the video meeting, click this link: https://meet.google.com/sqh-dsur-evv Otherwise, to join by phone, dial +1 413-752-0156 and enter this PIN: 999 769 115# To view more phone numbers, click this link: https://tel.meet/sqh-dsur-evv?hs=5

[rpachaly]

New infiltration editor ready!

• SSURGO data for estimating soil G&A parameters
• OSM data for estimating land use G&A parameters
• Documentation added for the new infiltration editor
• Tested on cities in Arizona, Nevada, and Colorado
• Minor bugs fixed

I pushed the branch.

[FLO-2DKaren]

@rpachaly I don't know what the rocket means. I assume it means launch.
I merged this branch into the regular plugin so I can give it to a couple of people to test.

I'll update the meta data too.