Questions regarding the configuration settings of pm_abcd_mrtm.ini

[wangshaobobetter]

The example model data is for a period of 31 years, but I want to run it using only ten years of data. However, I noticed that some values in the output directory are not on the same scale as the 31-year data, such as GCAMRegion_runoff_km3peryear_pm_abcd_mrtm_watch.csv. How should the configuration file be set? When I run it with the first ten years of the example data, I encounter the same issue, but running it with the full 31 years of data gives correct results。 I only modified the input files in the 'pet' and 'climate' folders, which are in the .npy format. I haven't made any changes to other input files. The dimensions used are 67420*120, representing ten years of data. Do I need to modify anything else? Thank you！ Below is the content of my configuration information file. [Project]

ProjectName = pm_abcd_mrtm_watch_1971_1980

RootDir = d:\work\xanthos-main1\example

InputFolder = input

OutputFolder = output

RefDir = reference pet_dir = pet

RoutingDir = routing

RunoffDir = runoff

DiagDir = diagnostics

AccWatDir = accessible_water

HydActDir = hydropower_actual

HistFlag = True

n_basins = 235

StartYear = 1971 EndYear = 1980

output_vars = pet, aet, q, soilmoisture, avgchflow

OutputFormat = 1

OutputUnit = 1

OutputInYear = 1

AggregateRunoffBasin = 1 AggregateRunoffCountry = 1 AggregateRunoffGCAMRegion = 1

PerformDiagnostics = 1

CreateTimeSeriesPlot = 0

CalculateDroughtStats = 1

CalculateAccessibleWater = 1

CalculateHydropowerPotential = 1

CalculateHydropowerActual = 1

Calibrate = 0

[PET]

pet_module = pm

[[penman-monteith]]

pet_dir = penman_monteith

pm_tas = tas_watch_monthly_degc_1971_1980.npy

pm_tmin = tasmin_watch_monthly_degc_1971_1980.npy

pm_rhs = rhs_watch_monthly_percent_1971_1980.npy

pm_rlds = rlds_watch_monthly_wperm2_1971_1980.npy

pm_rsds = rsds_watch_monthly_wperm2_1971_1980.npy

pm_wind = wind_watch_monthly_mpers_1971_1980.npy

pm_lct = lucc1901_2010_lump.npy

pm_nlcs = 8

pm_water_idx = 0

pm_snow_idx = 6

pm_lc_years = 1900, 1910, 1920, 1930, 1940, 1950, 1960, 1970, 1980, 1990, 2000, 2005, 2010

[Runoff]

runoff_module = abcd

[[abcd]]

runoff_dir = abcd

calib_file = pars_watch_1971_1990_decadal_lc.npy

runoff_spinup = 120

jobs = -1

TempMinFile = example/input/climate/pr_gpcc_watch_monthly_mmpermth_1971_1980.npy

PrecipitationFile =example/input/climate/pr_gpcc_watch_monthly_mmpermth_1971_1980.npy

[Routing]

routing_module = mrtm

[[mrtm]]

routing_dir = mrtm

routing_spinup = 120

channel_velocity = velocity_half_degree.npy

flow_distance = DRT_half_FDISTANCE_globe.txt

flow_direction = DRT_half_FDR_globe_bystr50.txt

[Diagnostics]

VICDataFile = vic_watch_hist_nosoc_co2_qtot_global_annual_1971_2000.nc WBMDataFile = wbm_qestimates.csv WBMCDataFile = wbmc_qestimates.csv UNHDataFile = UNH_GRDC_average_annual_1986_1995.nc

Scale = 0 [Drought]

threshold_start_year = 1971 threshold_end_year = 1980

threshold_nper = 12 [AccessibleWater]

ResCapacityFile = total_reservoir_storage_capacity_BM3.csv BfiFile = bfi_per_basin.csv

HistEndYear = 1980 GCAM_StartYear = 1971 GCAM_EndYear = 1980 GCAM_YearStep = 1

MovingMeanWindow = 9

Env_FlowPercent = 0.1 [HydropowerPotential]

hpot_start_date = "1/1971"

q_ex = 0.7

ef = 0.8 [HydropowerActual]

hact_start_date = "1/1971" [TimeSeriesPlot] Scale = 1 MapID = 999

[mengqi-z]

Hi @wangshaobobetter , I noticed the TempMinFile was set to read precipitation in your configuration file: TempMinFile = example/input/climate/pr_gpcc_watch_monthly_mmpermth_1971_1980.npy.

Could this be the reason for the incorrect results?

[wangshaobobetter]

嗨，我注意到在您的配置文件中设置为读取沉淀：。TempMinFile``TempMinFile = example/input/climate/pr_gpcc_watch_monthly_mmpermth_1971_1980.npy

这可能是错误结果的原因吗？

@mengqi-z Thank you for pointing out the issue. I have made the modification to TempMinFile = d:\work\xanthos-main\example\input/climate/.npy, but the error remains unchanged.

[mengqi-z]

Hi @wangshaobobetter, Could you explain more and give me an example regarding this?

some values in the output directory are not on the same scale as the 31-year data, such as GCAMRegion_runoff_km3peryear_pm_abcd_mrtm_watch.csv

Note that runoff is aggregated to different spatial scales (country, basin, and region). For example, the one you mentioned above is the aggregated runoff at regional scale (e.g., USA, China). So please compare the runoff at the same spatial scale when you are using different climate data.

You missed the file name in your modification TempMinFile = d:\work\xanthos-main\example\input/climate/.npy, in case you didn't notice. Otherwise, your configuration file looks correct. I would suggest double check your modified inputs and make sure their dimensions are the same (67420 X 120).

[wangshaobobetter]

@mengqi-z

Note that runoff is aggregated to different spatial scales (country, basin, and region). For example, the one you mentioned above is the aggregated runoff at regional scale (e.g., USA, China). So please compare the runoff at the same spatial scale when you are using different climate data.

You missed the file name in your modification TempMinFile = d:\work\xanthos-main\example\input/climate/.npy, in case you didn't notice. Otherwise, your configuration file looks correct. I would suggest double check your modified inputs and make sure their dimensions are the same (67420 X 120).

*您好，非常感谢您的回复，我做这个的目的是想跑自己找的10年的数据，数据已经进行过处理，保证维度是67420乘以120，十年的数据，且为了和模型其他数据对应好，还将数据按照示例数据的经纬度坐标也一一对应了，可是运行模型输出后的数据出入很大，例如output下的GCAMRegion_runoff_km3peryear_pm_abcd_mrtm_watch.csv中 我得到的数据都是两位数，如China 56多，但示例中的运行输出的是2095多，数量级不一样。 我还做了试验，就是将示例中的31年数据，都提取了前面十年的数据，即1971-1980，运行模型后输出的数据居然和我跑自己下载十年的数据是一个数量级，不到100，是我哪里配置信息设置错误了吗？目前我修改的输入数据仅是pet 和climate 的输入，其他的输入都没变动。配置文件可参考我刚开始提出的issue 的。

[mengqi-z]

I am not sure what the issue is without checking your inputs, because the configuration settings look correct to me. I suspect it might be because of the data format, or mislabeled unit. Could you post your logfile from running the 10-year data (the data you modified from the example data)? Thanks.

[wangshaobobetter]

I am not sure what the issue is without checking your inputs, because the configuration settings look correct to me. I suspect it might be because of the data format, or mislabeled unit. Could you post your logfile from running the 10-year data (the data you modified from the example data)? Thanks.

@mengqi-z 非常感谢您的耐心指导，下面是我跑示例数据的前十年1971_1980的日志，结果和跑示例数据1971_2001 结果不一样，如output下的GCAMRegion_runoff_km3peryear_pm_abcd_mrtm_watch.csv 1971年的china 数据，其他国家和区域也是这样的差别。

---

ProjectName : pm_abcd_mrtm_watch_1971_1980 InputFolder : example\input OutputFolder: example\output\pm_abcd_mrtm_watch_1971_1980 StartYear - End Year: 1971-1980 Number of Months : 120 Running: Historic Mode Diagnostics will be performed using the data file: example\input\diagnostics\vic_watch_hist_nosoc_co2_qtot_global_annual_1971_2000.nc ProjectName : pm_abcd_mrtm_watch_1971_1980 ProjectName : pm_abcd_mrtm_watch_1971_1980 InputFolder : example\input InputFolder : example\input OutputFolder: example\output\pm_abcd_mrtm_watch_1971_1980 OutputFolder: example\output\pm_abcd_mrtm_watch_1971_1980 StartYear - End Year: 1971-1980 StartYear - End Year: 1971-1980 Number of Months : 120 Number of Months : 120 Running: Historic Mode Running: Historic Mode Diagnostics will be performed using the data file: example\input\diagnostics\vic_watch_hist_nosoc_co2_qtot_global_annual_1971_2000.nc Diagnostics will be performed using the data file: example\input\diagnostics\vic_watch_hist_nosoc_co2_qtot_global_annual_1971_2000.nc ---Simulation in progress... ---Simulation in progress... Processing PET... Processing PET... Processing Years: 1971 to 1972 Processing Years: 1971 to 1972 Processing Years: 1972 to 1973 Processing Years: 1972 to 1973 Processing Years: 1973 to 1974 Processing Years: 1973 to 1974 Processing Years: 1974 to 1975 Processing Years: 1974 to 1975 Processing Years: 1975 to 1976 Processing Years: 1975 to 1976 Processing Years: 1976 to 1977 Processing Years: 1976 to 1977 Processing Years: 1977 to 1978 Processing Years: 1977 to 1978 Processing Years: 1978 to 1979 Processing Years: 1978 to 1979 Processing Years: 1979 to 1980 Processing Years: 1979 to 1980 Processing Years: 1980 to 1980 Processing Years: 1980 to 1980 PET processed in 63.02931261062622 seconds--- PET processed in 63.02931261062622 seconds--- Processing Runoff... Processing Runoff... Processing spin-up and simulation for basins 1...235 Processing spin-up and simulation for basins 1...235 Runoff processed in 1.6805241107940674 seconds--- Runoff processed in 1.6805241107940674 seconds--- Processing Routing... Processing Routing... Routing processed in 156.20701098442078 seconds--- Routing processed in 156.20701098442078 seconds--- ---Simulation has finished successfully: 220.92386412620544 seconds --- ---Simulation has finished successfully: 220.92386412620544 seconds --- ---Start Accessible Water: ---Start Accessible Water: ---Accessible Water has finished successfully: 1.5902864933013916 seconds ------ ---Accessible Water has finished successfully: 1.5902864933013916 seconds ------ ---Start Drought Statistics: ---Start Drought Statistics: Calculating drought thresholds Calculating drought thresholds ---Drought Statistics has finished successfully: 0.14320611953735352 seconds ------ ---Drought Statistics has finished successfully: 0.14320611953735352 seconds ------ ---Start Hydropower Potential: ---Start Hydropower Potential: ---Hydropower Potential has finished successfully: 4.589032173156738 seconds ------ ---Hydropower Potential has finished successfully: 4.589032173156738 seconds ------ ---Start Hydropower Actual: ---Start Hydropower Actual: ---Hydropower Actual has finished successfully: 21.36350727081299 seconds ------ ---Hydropower Actual has finished successfully: 21.36350727081299 seconds ------ ---Start Diagnostics: ---Start Diagnostics: ---Diagnostics has finished successfully: 0.11775565147399902 seconds ------ ---Diagnostics has finished successfully: 0.11775565147399902 seconds ------ ---Output simulation results: ---Output simulation results: Outputting data annually Outputting data annually Unit is km3peryear Unit is km3peryear pet output dimension is (67420, 10) pet output dimension is (67420, 10) aet output dimension is (67420, 10) aet output dimension is (67420, 10) q output dimension is (67420, 10) q output dimension is (67420, 10) soilmoisture output dimension is (67420, 10) soilmoisture output dimension is (67420, 10) avgchflow output dimension is (67420, 10) avgchflow output dimension is (67420, 10) Aggregating by Basin Aggregating by Basin Aggregating by Country Aggregating by Country Aggregating by GCAM Region Aggregating by GCAM Region Aggregated unit is km3peryear Aggregated unit is km3peryear ---Output finished: 4.79646635055542 seconds --- ---Output finished: 4.79646635055542 seconds --- End of pm_abcd_mrtm_watch_1971_1980 End of pm_abcd_mrtm_watch_1971_1980

---

有几个问题想和您探讨： 1.我们运行模型修改的数据有：气候数据和Pet 下的相关npy 类型的数据，我们注意到模型中，notebooks\quick_starter.ipynb里，runoff数据是随机生成，所以我们还想换掉runoff，除了气象数据、PET和runoff数据外，其他的数据就用模型的示例数据了，ReadMe_Input_Data.txt数据来源于真实研究的数据结果，这样运行模型的方式是否科学？ 2.我们注意到ReadMe_Input_Data.txt提供的runoff 数据的参考来源为：Original Data can be accessed at http://daac.ornl.gov/cgi-bin/dsviewer.pl?ds_id=994 120+2 columns: longitude, latitude, monthly runoff (mm/mon) from 1986-1995. 时间较为久远，且没有读取到这个数据集的经纬度信息，请问是否有适合于xanthos模型的更新格网尺度的径流数据来源？ 我们最后实际想运行accessible water 模块，相关的runoff 也想用实际的，不想用模型随机生成的方式 3.您是否用过xanthos 模型的输出数据作为GCAM的输入，如streamflow 非常感谢您的帮助！并希望与您取得进一步联系，能否留下个Email?

[mengqi-z]

There is something weird happening because each logging information is repeating twice. I did a test with 20-year data and it ran without issue using the same configuration settings you have. So my guess is that your input climate data might have issues. The data format might be wrong. Note that npy file should NOT have grid id in it.

Regarding your other questions:

1. The runoff created as random numbers in notebooks\quick_starter.ipynb is just an quick example because we didn't provide monthly runoff output in the example folder, but we want to show how to use Xanthos modules, which needs monthly runoff. When you run Xanthos, you can choose to output monthly runoff.
2. These are runoff data we used to validate Xanthos. The links might be outdated, some of them are not accessible anymore. You can find your own runoff observations to calibrate Xanthos simulated runoff. Currently, the default calibration data is VIC runoff. However, the default configuration setting does not perform calibration. We provided the calibrated parameters, so if you didn't turn "calibration" on, then Xanthos will use these optimal parameters to simulate runoff.
3. Yes. You can contact me at mengqi.zhao@pnnl.gov

[wangshaobobetter]

@mengqi-z 你好，非常感谢您的耐心指导，我给您发了邮件，将一个输入数据的文件发给您了，有时间烦请您查看下数据的格式是否有问题，不胜感激。

[wangshaobobetter]

@mengqi-z 你好，非常感谢您的指导，我查看过我的数据，没发现问题，我看到您之前跑过20数据的例子，能否给我发一个hurs的20年数据文件，给你邮箱发邮件了，谢谢您。

[wangshaobobetter]

got the result I wanted