How to prepare HSWD?

[joelferg]

I'm trying to prepare the HSWD for use in Module 4, but a few things in the documentation relating to the soil characteristic CSVs are confusing me.

1) Does SPH (soil phase rating) refer to PHASE1 in the database? If so, is it actually 0/1 coded as mentioned in Table 3? From Figure 2 it appears to be a string field.

2) What do the SPR, OSD, and VSP columns correspond to in the database?

3) Is there any particular way things should be aggregated when multiple rows correspond to a given mapping unit?

Thank you in advance.

[lixuan-code]

I encountered exactly the same problem as yours, I am also confused about this problem, have you solved this problem?

[NokuzaSezo97]

Dear @joelferg and @xfry-lixuan,

This is Swun Wunna Htet, a developer of PyAEZ system. I'm sorry for the late reply as I found out this question list today. Here are the information for the information:

1. Yes, SPH refers to PHASE1 column of HWSD database. The definition of each number for Phase is within the database sheet called "D_PHASE". Use the string information from that data into your csv excel sheet.

2. OSD refers to obstacles to roots and impermeable layers. In HWSD database, it refers to column name "ROOTS". ROOTS have seven unique indices which you can find the definition in "D_ROOTS" sheet.

3. SPR and VSP share the same column named "ADD_PROP" in HWSD database. Even though they might share same column properties for a specific SMU for csv file, the reduction factor setting for each will be more or less different , because SPR is considered for SQ3 (Rooting conditions) and VSP for SQ7 (Soil Workability). For that, you might need to change reduction factor python file for those two properties.

4. If you encounter multiple rows for a single SMU in HWSD database, one must judge which row would be ideal for to represent. As far as I've concerned, there are additional columns (such as soil or no-soil column, SHARE, source of soil data etc.) that might be helpful for your row selection or reviewing soil type might also help as well. My example judgement (I might be wrong as I'm not a soil expert, so kindly bear with me) : If I have a SMU with multiple rows which one of them have more share of soil unit percentages, I would choose that row as I considered that SMU having less percentage is less likely to reflect their soil properties, or if I have a SMU that is not soil, I would have this SMU pixels excluded out in calculation). So, for that, it's your call.

The reply is long, but I do hope this helps in PyAEZ data preparation. And thank you so much for your interest in utilizing PyAEZ. Please don't hesitate to suggest for any clarification from us.

Regards,

Swun Wunna Htet

[lixuan-code]

When I run NB4 with my own prepared soil feature table, I get the following error, but I can't solve the problem, it doesn't seem to be a csv format problem. So I would like to ask you, can you answer it conveniently?

`--------------------------------------------------------------------------- StopIteration Traceback (most recent call last) Cell In[6], line 7 2 '''calculate soil qualities and ratings and applying soil constraints - Rain-fed''' 3 # Current implementation of creating two csv files is to look into the metadata of HWSD database and copy the required parameters 4 # into the files. Depending how many soil types in your area of interest, the list may either be short or long ----> 7 soil_constraints.calculateSoilQualities('R', topsoil_path='./mydata_input/TOP_SOIL.csv', 8 subsoil_path='./mydata_input/SUB_SOIL.csv') # I: Irrigated, R: Rain-fed 10 # L: Low, I: Intermediate , H: High input 11 soil_constraints.calculateSoilRatings('I')

File /media/sf_Linux/PyAEZ-master/./pyaez/SoilConstraints.py:136, in SoilConstraints.calculateSoilQualities(self, irr_or_rain, topsoil_path, subsoil_path) 134 name_list = [] 135 for row_top in top_reader: --> 136 row_sub = next(sub_reader) 138 row_count = row_count + 1 139 if row_count == 1:

StopIteration: `

[NokuzaSezo97]

Dear @xfry-lixuan , could you check whether the subsoil csv sheet has data or else? This error might occur when excel sheet has column names but you have no elements in your sheet, which means at least a row of SMU properties must exist in that sheet. If that's not the case, if you could send me both the excel sheets to me, I'll be able to investigate the potential system bug or something. Thanks.

[lixuan-code]

Thank you! I'm very sorry for replying to you so late. The issue with HWSD mentioned above has been resolved due to the inability of soil characteristics to have both topsoil and subsoil.

[JCAELA]

Hello Swun,

I followed the instructions to prepare the two csv files for soil proprieties but I get an error.

I checked the files carefully and I could not find what is wrong with it. Can I share the files with you so you can see what is wrong with the files?

Thank you for your support.

[NokuzaSezo97]

Dear @JCAELA,

I would suggest to check on the input csv columns' data types before providing the file path. I suspect that the confusion of reading csv file due to wrong data type setting in sheet could cause the problem. If the problem persists, kindly provide me an example csv sheet you're working on it for potential bug fixes.

Regards,

Swun Wunna Htet

[JCAELA]

Hi Swun,

I followed the instructions for the soil data preparation and compared my files with the samples found in the tutorial and I still got error and still could not find what is wrong.

I am sending you the files. soil_characteristics_subsoil.csv soil_characteristics_topsoil.csv

Thank you

José Caela

[NokuzaSezo97]

Dear @JCAELA,

I checked your csv files you provided and the error. The values are to be set as structured excel sheet, for individual column cells, not unstructured. I'm sending two images which are correct and incorrect. Soil calculation works with excel columns, each column refers to specific soil characteristics. Please rearrange your excel sheet column setting correctly according to the image. Once you've readjusted, it will be fine.

Correct way of value setting.

Incorrect way of value setting.

Regards,

Swun Wunna Htet

[joelferg]

@NokuzaSezo97

Thank you Swun, your explanation was helpful. I'm just getting back to this project now and the one thing I'm still a bit unclear on is how to make SPR and VSP. Do they correspond to particular values of ADD_PROP.

Thanks,

Joel

[NokuzaSezo97]

Dear @joelferg,

By GAEZ v4 model reference, VSP and SPH share the same definition of "soil having vertic properties or soil type vertisol". In the ALL_REDUCTION_FACTOR files, I mistakenly defined VSP as vertical properties (which was not my intention), my apologies for this confusion.

As I said, they share same definition but if you look at the source code or in GAEZ v4 model documentation, they are used for different soil quality calculation (SQ 7 for VSP and SQ3 for SPR) and the factors are input/management-level specific. So, the value setting is up to your judgement. Just be sure all reductions factors are set up in descending order from left to right.

Also, I really appreciate for your debugging contribution to some source codes I've overlooked. I'll confirm to merge in master branch. Thank you very much.

Warm regards,

Swun Wunna Htet

[joelferg]

Ahhh, sorry I think I just misunderstood what you said before. They literally correspond to the same column in HSWD with no transformation necessary. Thank you Swun.

And of course! I've found a couple more things of that nature and will make another pull request soon.