search

USDA-ARS-NWRC / awsm

Automated Water Supply Model (AWSM) was developed at the USDA Agricultural Research Service. AWSM was designed to streamline the workflow used to forecast the water supply of multiple water basins.
Other
9 stars 2 forks source link

Topocalc v0.5 #89

Closed scotthavens closed 3 years ago

scotthavens commented 3 years ago

SMRF updated to topocalc v0.5 which requires updating gold files.

Additionally, moved SMRF dependency to the default master branch to keep the HEAD of AWSM and SMRF together.

scotthavens commented 3 years ago

The error seems to be stemming from SMRF for test_awsm_lakes.TestLakesSMRFiPysnobalThreadedHRRR:

ERROR:smrf.utils.queue.data_wind_speed:Timeout occurred while retrieving an item at 2019-10-01 16:00:00+00:00 from queue
ERROR:smrf.utils.queue.data_wind_speed:
ERROR:smrf.utils.queue.storm_days:Timeout occurred while retrieving an item at 2019-10-01 16:00:00+00:00 from queue
ERROR:smrf.utils.queue.storm_days:
ERROR:smrf.utils.queue.albedo_ir:Timeout occurred while retrieving an item at 2019-10-01 16:00:00+00:00 from queue
ERROR:smrf.utils.queue.albedo_ir:
ERROR:smrf.utils.queue.output:Timeout occurred while retrieving an item at 2019-10-01 16:00:00+00:00 from queue
ERROR:smrf.utils.queue.output:
ERROR:smrf.utils.queue.data_wind_speed:Timeout occurred while retrieving an item at 2019-10-01 16:00:00+00:00 from queue
ERROR:smrf.utils.queue.data_wind_speed:
ERROR:smrf.utils.queue.net_solar:Timeout occurred while retrieving an item at 2019-10-01 16:00:00+00:00 from queue
ERROR:smrf.utils.queue.net_solar:
ERROR:smrf.utils.queue.wind_speed:Timeout occurred while retrieving an item at 2019-10-01 16:00:00+00:00 from queue
ERROR:smrf.utils.queue.wind_speed:
jomey commented 3 years ago

Should we cut a new release for SMRF to get this in?

scotthavens commented 3 years ago

Should we cut a new release for SMRF to get this in?

Let me think about this. I'm not liking how many patches we're doing to get SMRF into AWSM, it was a good idea in theory but we're creating a lot of small releases. Seems like AWSM main should be linked to SMRF master? Then they'd get released together?

jomey commented 3 years ago

I would not worry about a lot of small releases with X.X.X. This was a bug fix and justifies a new release.

Take any chromium based browser for instance. There the last number grows quite fast as well.

Once there is a sync between AWSM and SMRF I would bump it to X.X.0

scotthavens commented 3 years ago

I understand in the software world, patch bumps are everywhere. But we're trying to make sure that these aren't stable releases but development releases. That is the big picture thinking.

For this, lets just bump and we'll figure that out later.

jomey commented 3 years ago

Isn't the 'stable' release also clearly communicated with the README? There it says 0.10 is the last stable. My expectation as a consumer would then be that 0.11 is the next and everything else in between is not

scotthavens commented 3 years ago

Isn't the 'stable' release also clearly communicated with the README? There it says 0.10 is the last stable. My expectation as a consumer would then be that 0.11 is the next and everything else in between is not

One can only hope! I'd like to see the stable version of SMRF get released to pypi under smrf, then there would be a good distinction between smrf and smrf-dev packages

jomey commented 3 years ago

I'd like to see the stable version of SMRF get released to pypi under smrf, then there would be a good distinction between smrf and smrf-dev packages

That could be another approach too.

Another one would be to have AWSM and SMRF always built against HEAD of master and then release with a frozen version in the requirements.txt. You can even extend that to all modules (WFR, katana). Would get rid of any version dances in development

scotthavens commented 3 years ago

That was what I was kinda leaning towards, just keep everything at HEAD and freeze when we want something stable. Trying to simplify the development and deployment.

jomey commented 3 years ago

May I double dog dare you and try this approach with this PR, replacing smrf-dev with the git url and see how it goes?

scotthavens commented 3 years ago

image

scotthavens commented 3 years ago

@jomey did you want to run goldmiester to document the changes?

jomey commented 3 years ago

I can run them ..

jomey commented 3 years ago

Documenting the difference to the current gold files in the main branch:

RME

ipysnobal nc_cold_content ipysnobal nc_evaporation ipysnobal nc_latent_heat ipysnobal nc_liquid_water ipysnobal nc_net_radiation ipysnobal nc_sensible_heat ipysnobal nc_snow_density ipysnobal nc_snowmelt ipysnobal nc_snow_soil ipysnobal nc_specific_mass ipysnobal nc_sum_energy_balance ipysnobal nc_surface_water_input ipysnobal nc_temperature_snowcover ipysnobal nc_temperature_surface ipysnobal nc_thickness ipysnobal nc_water_saturation

Lakes

ipysnobal nc_cold_content ipysnobal nc_evaporation ipysnobal nc_latent_heat ipysnobal nc_liquid_water ipysnobal nc_net_radiation ipysnobal nc_sensible_heat ipysnobal nc_snow_density ipysnobal nc_snowmelt ipysnobal nc_snow_soil ipysnobal nc_specific_mass ipysnobal nc_sum_energy_balance ipysnobal nc_temperature_lower ipysnobal nc_temperature_snowcover ipysnobal nc_temperature_surface ipysnobal nc_thickness ipysnobal nc_thickness_lower ipysnobal nc_water_saturation

jomey commented 3 years ago

Looks like all changes are within the expected variables related to energy balance terms that are affected by the updates TopoCalc clear sky changes. RME shows a stronger change for sum_energy_balance and net_radiation compared to Lakes.

scotthavens commented 3 years ago

Looks like all changes are within the expected variables related to energy balance terms that are affected by the updates TopoCalc clear sky changes. RME shows a stronger change for sum_energy_balance and net_radiation compared to Lakes.

Looks like it's the other way around, Lakes had a larger change in the EB terms due to the linear artifacts that were more prevalent.

jomey commented 3 years ago

Looks like it's the other way around, Lakes had a larger change in the EB terms due to the linear artifacts that were more prevalent.

:grinning: That's what I meant