search

ncss-tech / jNSMR

R frontend for the Java Newhall Simulation Model (jNSM) -- "A Traditional Soil Climate Simulation Model"
https://ncss-tech.github.io/jNSMR
BSD 3-Clause "New" or "Revised" License
1 stars 0 forks source link

"Undefined" classes of soil moisture regimes and their subdivisions #8

Closed brownag closed 6 months ago

brownag commented 1 year ago

@dylanbeaudette pointed out an example of an extent (Coweeta) where all grid cells return "Perudic" moistureRegime with NA regimeSubdivision1 and regimeSubdivision2.

This has been partially addressed in https://github.com/ncss-tech/jNSMR/commit/66c29b4251110b6fa390c2ce1741c15b5ace6f29 which modifies the factor levels assigned when there is a valid SMR but no subdivision. The " " values returned are converted to "Undefined" on R side for consistency in SpatRaster method regardless of extent. You get expected values when you use a larger (e.g. 10x coweeta) extent (that has more variation in SMR as well as defined subdivisions)

For a more general fix might need some adjustment in the Java code to make sure the values returned when the subdivision is "Undefined" are consistent. These values appear to be missing (NA) in the case where that is the only level, which is unexpected. Possibly a degenerate case in the Java batching method used by jNSMR:::batch2()

The " " values returned are converted to "Undefined" on R side for consistency, and this is what you get when you use a larger extent that has more variation in SMR as well as defined subdivisions

brownag commented 1 year ago

Turns out the problem columns were coming back as NA due to missing columns in several of the individual core results. Still need to investigate how this happens, but something has been overlooked when those fields are read out of the org.psu.newhall.sim.NewhallBatchResults class fields

rbindlist(..., fill=TRUE) is used to re-construct a data.frame that corresponds to the (non-NA AWS) portion of the block that is read by terra... and that is where the NA values were sneaking in.

Before MicrosoftTeams-image

After image

dylanbeaudette commented 1 year ago

Tried the latest version, and now some new (odd) results.

image

This was using the Coweeta grids (UTM z17) I shared and the "original" PRISM data that are used by the MU comparison reports (WGS84).

brownag commented 1 year ago

Considering I have run it on that extent and not encountered this issue... I am gonna need some more specific information on the problem grids and the code you are running to pass them to the model

dylanbeaudette commented 1 year ago

Sure, it must have something to do with resampling then. All data + code:

S:\NRCS\Archive_Dylan_Beaudette\temp\coweeta

brownag commented 1 year ago

OK, I can reproduce this highly unexpected result with the newhall-simulation.R file and my newhall-simulation2.R file. I'll need to dig into this further; wondering if there is an issue with the recoding of NA values in the latest terra (in response to my recent issue with categorical rasters). It may be that I broke something else in my recent fixes, but can confirm that I see what you see

brownag commented 1 year ago

OK, so with CRAN terra 1.6-17 I get something pretty close to" expected," but still not 100%. I'll undo some of the things I did and see if I can fix those NULL columns that sneak in there image

image

brownag commented 1 year ago

image OK, the blocking is "fixed" after https://github.com/ncss-tech/jNSMR/commit/3205f089236dde60361f5d3405988223e053af59. These are made with your input data/script and a modified version of terra 1.6-28; should also work with the current CRAN terra. Now to try and simplify the issue and identify the core problem/need for a new issue in terra.

dylanbeaudette commented 1 year ago

Thanks. I'll be back to this on Tuesday.