netcdf writing issues - Githubissues

kbagstad commented 13 years ago

Ciao Ferd -

Gary ran flood & sediment models for Puget Sound and was able to get them running. However, the netcdf's are essentially empty (zero values for nearly all layers) - possibly either ontology issues (though the identification models run and display just fine) or netCDF writing issues. I don't understand how the ontology issues could keep causing problems, because the outputs of the identification models worked fine - it's just when we add flood results that they become deeply screwy. Also Gary added values for the non-displaying sediment layers to the colormaps-properties file and was able to get them to display fine in the identificaiton statement model -o.

Check out .nc files at raid/nc_outputs/USSEE/flood-regulation-residents-100-puget1024-downscale3.nc and sediment-reservoirs-puget1024-downscale3.nc - this should show zero values for most results, with a few odd exceptions like possible sediment flow (see emailed image).

Gary's suggestion was that we either gut and rewrite the ontologies (in a suspect Protege environment that guarantees very little in the way of robust ontologies) or add everything to colormaps-properties - neither appealing options. Any thoughts much appreciated...

Thanks, Ken

fvilla commented 13 years ago

You're saying that the values you don't see are those from the flow models only? If so, they are not classified, they're numbers so the ontology is not an issue there. Unfortunately I cannot run anything on this machine - flow models OR IDV so I can't even check the netcdf. And nobody buys me ARC which I'm pretty sure wouldn't run either. If only the flow model results are screwy, then it's probably related to a problem Gary reported to me yesterday - uncoordinated changes between flow and thinklab API that made the computation of distribution parameters wrong. He claims he checked and fixed some of my code so that may have changed. Unfortunately unless you point me to a model/context that runs in little memory (possibly with some absurd downscaling factor) and reproduces that, I can do very little until I get home.

lambdatronic commented 13 years ago

Correct. The maps that are coming back all zero are from my flow model outputs. However (and this is a big however), this is not a generic interface problem between my code and yours. I ran the aesthetic view and proximity models, and their results came out perfectly. I know (based on my log messages) that the flood and sediment models ran to completion and made lots and lots of non-zero computations (~200,000 non-zero carriers for both models). No errors were generated during the model running or model result to observation translation process that your code does in SPANDistributionState.

Furthermore (and this is absolutely amazing), as Ken points out above, the possible sediment flow map did have real values, while everything else was zero. Now this is obviously impossible, so some kind of magical weirdness must be happening on the way to the netCDFs. I'll try the models using -v and -d and comment again when I've seen something further.

Remember, it's the flood and sediment models that are all zeros, while views and proximity create properly visualizable results. So it's localized to those models.

Bah.

kbagstad commented 13 years ago

Gary, I'm assuming you're able to re-generate results with zeroes in new netCDF's and view the zeroes in grass? I've seen the zeroes in both arc and IDV so I'm not doubting the results, but replication is never bad...

fvilla commented 13 years ago

2 questions - do the zeros appear with -d too, and can you send me a command to replicate that using a model that is downscaled enough to run on a toy computer? thanks

kbagstad commented 13 years ago

Zeroes do not appear with -d.

I tried to run these at small spatial extents to get something Ferd might be able to hit on a laptop, and in both instances was left with mysterious errors after the model had gotten to the stage of running the span models. Here are the commands and errors, for Gary's amusement:

model -o foo.nc core.models.flood-puget/flood-regulation-residents-100 core.contexts.beta/viewshed256.nc

We are inside runSPAN!
Unpacking observation into data-layers.
Extracting Precipitation layer.
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource MD[floodService:                Precipitation: ]>
It's deterministic.
Extracting FloodSink layer.
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource MD[floodService:                FloodSink: ]>
It's deterministic.
Extracting FloodResidentsUse100 layer.
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource MD[floodService:                FloodResidentsUse100: ]>
It's deterministic.
Extracting flow layers: (Altitude River Floodplains100 Levees)
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource MD[geophysics:Al                titude: ]>
It's deterministic.
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource MD[geofeatures:R                iver: ]>
It's deterministic.
Inside unpack-datasource!
Checking datasource type... #<MemObjectContextualizedDatasource MO[floodService:                Floodplains100: ]>
It's probabilistic and continuous.
Application start failed.
java.lang.RuntimeException: java.lang.NullPointerException
        at clojure.lang.LazyCons.first(LazyCons.java:44)
        at clojure.lang.RT.first(RT.java:539)
        at clojure.core$first__2743.invoke(core.clj:40)
        at clojure.core$into__3590.invoke(core.clj:1620)
        at clj_span.aries_span_bridge$layer_map_from_observation__5891.invoke(ar                ies_span_bridge.clj:262)
        at clj_span.aries_span_bridge$span_driver__5901.invoke(aries_span_bridge                .clj:298)
at aries$get_span_proxy__4050$fn__4052.invoke(aries-core.clj:25)
        at clojure.proxy.java.lang.Object$SPANProxy.runSPAN(Unknown Source)
        at org.integratedmodelling.aries.core.implementations.observations.SPANT                ransformer.transform(SPANTransformer.java:161)
        at org.integratedmodelling.corescience.context.ObservationContext.proces                sTransformations(ObservationContext.java:719)
        at org.integratedmodelling.corescience.context.ObservationContext.proces                sTransformations(ObservationContext.java:637)
        at org.integratedmodelling.corescience.context.ObservationContext.run(Ob                servationContext.java:758)
        at org.integratedmodelling.corescience.ObservationFactory.contextualize(                ObservationFactory.java:73)
        at org.integratedmodelling.modelling.model.ModelFactory$ContextualizingM                odelResult.getResult(ModelFactory.java:253)
        at org.integratedmodelling.modelling.commands.ModelCommand.execute(Model                Command.java:132)
        at org.integratedmodelling.thinklab.command.CommandManager.submitCommand                (CommandManager.java:144)
        at org.integratedmodelling.thinklab.commandline.Shell.startConsole(Shell                .java:114)
        at org.integratedmodelling.thinklab.shell.ShellApplication.startApplicat                ion(ShellApplication.java:130)
        at org.java.plugin.boot.Boot.boot(Boot.java:346)
        at org.java.plugin.boot.Boot.main(Boot.java:243)
Caused by: java.lang.NullPointerException
        at clojure.lang.Reflector.invokeNoArgInstanceMember(Reflector.java:253)
        at clj_span.aries_span_bridge$fn__5833.invoke(aries_span_bridge.clj:187)
        at clojure.lang.MultiFn.invoke(MultiFn.java:156)
        at clj_span.aries_span_bridge$layer_map_from_observation__5891$fn__5896.                invoke(aries_span_bridge.clj:263)
        at clojure.core$map__3371$fn__3374.invoke(core.clj:1278)
        at clojure.lang.LazyCons.first(LazyCons.java:40)
        ... 19 more

And the error from the sediment model:

model -o foo.nc core.models.sediment-puget/sediment-farmers core.contexts.beta/viewshed256.nc

Unpacking observation into data-layers.
Extracting SedimentSourceValueAnnual layer.
Inside unpack-datasource!
Checking datasource type... #<MemObjectContextualizedDatasource MO[soilretentionEcology:Sediment                                SourceValueAnnual: ]>
It's probabilistic and continuous.
Breakpoints:   (0.0 0.01 30000.0 100000.0 300000.0)
Example Probs: (0.04553249999999998 0.16198749999999998 0.206565 0.585915)
Extracting AnnualSedimentSink layer.
Inside unpack-datasource!
Checking datasource type... #<MemObjectContextualizedDatasource MO[soilretentionEcology:AnnualSe                                dimentSink: ]>
It's probabilistic and continuous.
Breakpoints:   (0.0 0.01 10000.0 20000.0 30000.0)
Example Probs: (0.5893185999999999 0.28827140000000007 0.08616700000000001 0.036243)
Extracting DepositionProneFarmers layer.
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource MD[soilretentionEcology:Depositi                                onProneFarmers: ]>
It's deterministic.
Extracting flow layers: (Altitude Floodplains Levee River)
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource MD[geophysics:Altitude: ]>
It's deterministic.
Inside unpack-datasource!
Checking datasource type... #<MemObjectContextualizedDatasource MO[soilretentionEcology:Floodpla                                ins: ]>
It's probabilistic and continuous.
Application start failed.
java.lang.RuntimeException: java.lang.IllegalArgumentException: No matching field found: getRang                                es for class org.integratedmodelling.thinklab.owlapi.Concept
        at clojure.lang.LazyCons.first(LazyCons.java:44)
        at clojure.lang.RT.first(RT.java:539)
        at clojure.core$first__2743.invoke(core.clj:40)
        at clojure.core$into__3590.invoke(core.clj:1620)
        at clj_span.aries_span_bridge$layer_map_from_observation__5891.invoke(aries_span_bridge.                                clj:262)
at clj_span.aries_span_bridge$span_driver__5901.invoke(aries_span_bridge.clj:298)
        at aries$get_span_proxy__4050$fn__4052.invoke(aries-core.clj:25)
        at clojure.proxy.java.lang.Object$SPANProxy.runSPAN(Unknown Source)
        at org.integratedmodelling.aries.core.implementations.observations.SPANTransformer.trans                                form(SPANTransformer.java:161)
        at org.integratedmodelling.corescience.context.ObservationContext.processTransformations                                (ObservationContext.java:719)
        at org.integratedmodelling.corescience.context.ObservationContext.processTransformations                                (ObservationContext.java:637)
        at org.integratedmodelling.corescience.context.ObservationContext.run(ObservationContext                                .java:758)
        at org.integratedmodelling.corescience.ObservationFactory.contextualize(ObservationFacto                                ry.java:73)
        at org.integratedmodelling.modelling.model.ModelFactory$ContextualizingModelResult.getRe                                sult(ModelFactory.java:253)
        at org.integratedmodelling.modelling.commands.ModelCommand.execute(ModelCommand.java:132                                )
        at org.integratedmodelling.thinklab.command.CommandManager.submitCommand(CommandManager.                                java:144)
        at org.integratedmodelling.thinklab.commandline.Shell.startConsole(Shell.java:114)
        at org.integratedmodelling.thinklab.shell.ShellApplication.startApplication(ShellApplica                                tion.java:130)
        at org.java.plugin.boot.Boot.boot(Boot.java:346)
        at org.java.plugin.boot.Boot.main(Boot.java:243)
Caused by: java.lang.IllegalArgumentException: No matching field found: getRanges for class org.                                integratedmodelling.thinklab.owlapi.Concept
        at clojure.lang.Reflector.getInstanceField(Reflector.java:235)
        at clojure.lang.Reflector.invokeNoArgInstanceMember(Reflector.java:257)
        at clj_span.aries_span_bridge$fn__5833.invoke(aries_span_bridge.clj:187)
        at clojure.lang.MultiFn.invoke(MultiFn.java:156)
        at clj_span.aries_span_bridge$layer_map_from_observation__5891$fn__5896.invoke(aries_spa                                n_bridge.clj:263)
        at clojure.core$map__3371$fn__3374.invoke(core.clj:1278)
        at clojure.lang.LazyCons.first(LazyCons.java:40)
        ... 19 more

lambdatronic commented 13 years ago

Here is the output of the sediment-farmers SPAN model run in a pretty small context. The results aren't going to be remotely interesting, since at this resolution, the stream networks are fragmented, but it suffices to show that all of the output layers should be non-zero. You should be able to replicate this command on your machine. I'm running it on my 2-core/4GB RAM laptop with JAVA_OPTS="-Xms512M -Xmx3072M -XX:PermSize=128M -XX:MaxPermSize=512M", but you could give it less memory, and it should still work fine.

> model -d core.models.sediment-puget/sediment-farmers core.contexts.beta/wria9_256

... lots of INFO statements ...

We are inside runSPAN!
Unpacking observation into data-layers.
Extracting SedimentSourceValueAnnual layer.
Inside unpack-datasource!
Checking datasource type... #<MemObjectContextualizedDatasource 
MO[soilretentionEcology:SedimentSourceValueAnnual: ]>
It's probabilistic and continuous.
Breakpoints:   (0.0 0.01 30000.0 100000.0 300000.0)
Example Probs: (0.9455 0.030000000000000002 0.019499999999999997 
0.004999999999999999)
Extracting AnnualSedimentSink layer.
Inside unpack-datasource!
Checking datasource type... #<MemObjectContextualizedDatasource 
MO[soilretentionEcology:AnnualSedimentSink: ]>
It's probabilistic and continuous.
Breakpoints:   (0.0 0.01 10000.0 20000.0 30000.0)
Example Probs: (0.5870500000000001 0.29850000000000004 0.07960000000000002 
0.034850000000000006)
Extracting DepositionProneFarmers layer.
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource 
MD[soilretentionEcology:DepositionProneFarmers: ]>
It's deterministic.
Extracting flow layers: (Altitude Floodplains Levee River)
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource 
MD[geophysics:Altitude: ]>
It's deterministic.
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource 
MD[soilretentionEcology:Floodplains: ]>
It's deterministic.
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource 
MD[infrastructure:Levee: ]>
It's deterministic.
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource 
MD[geofeatures:River: ]>
It's deterministic.
Flow Parameters:
flow-model         = SedimentTransport
downscaling-factor = 3
rv-max-states      = 10
source-threshold   = 1000.0
sink-threshold     = 500.0
use-threshold      = 0.0
trans-threshold    = 100.0
cell-width         = 341.0322037982669 meters
cell-height        = 560.2763159253495 meters
source-type        = :finite
sink-type          = :finite
use-type           = :infinite
benefit-type       = :rival
result-type        = :closure-map
value-type         = :varprop
animation?         = false
save-file          = nil
(Pausing 10 seconds)
Preprocessing the input data layers.
Resampling matrix from 118 x 256 to 39 x 85...
  Distinct Layer Values: [Pre] 1722 [Post] 3201
Zeroing layer below 1000.0...
  Distinct Layer Values: [Pre] 3201 [Post] 3189
Resampling matrix from 118 x 256 to 39 x 85...
  Distinct Layer Values: [Pre] 20 [Post] 638
Zeroing layer below 500.0...
  Distinct Layer Values: [Pre] 638 [Post] 528
Resampling matrix from 118 x 256 to 39 x 85...
  Distinct Layer Values: [Pre] 2 [Post] 30
Zeroing layer below 0.0...
  Distinct Layer Values: [Pre] 30 [Post] 30
Resampling matrix from 118 x 256 to 39 x 85...
  Distinct Layer Values: [Pre] 1801 [Post] 3212
Resampling matrix from 118 x 256 to 39 x 85...
  Distinct Layer Values: [Pre] 2 [Post] 662
Resampling matrix from 118 x 256 to 39 x 85...
  Distinct Layer Values: [Pre] 2 [Post] 1102
Resampling matrix from 118 x 256 to 39 x 85...
  Distinct Layer Values: [Pre] 2 [Post] 256

Running SedimentTransport flow model.
Source points: 3188
Sink points:   569
Use points:    31
Finding nearest stream points...done. [Shifted 438 ids]
Computing flood activation factors...
|========================>| Completed 438 of 438 (100.0%)
All done.
Moving the sediment carriers downhill and downstream...
Carriers:       2963 | On Land:       1608 | In Stream:       1355
Carriers:       2803 | On Land:        694 | In Stream:       2109
Carriers:       1963 | On Land:        268 | In Stream:       1695
Carriers:       1267 | On Land:        123 | In Stream:       1144
Carriers:        836 | On Land:         69 | In Stream:        767
Carriers:        564 | On Land:         42 | In Stream:        522
Carriers:        391 | On Land:         30 | In Stream:        361
Carriers:        268 | On Land:         22 | In Stream:        246
Carriers:        170 | On Land:         11 | In Stream:        159
Carriers:        104 | On Land:          9 | In Stream:         95
Carriers:         63 | On Land:          6 | In Stream:         57
Carriers:         42 | On Land:          4 | In Stream:         38
Carriers:         27 | On Land:          3 | In Stream:         24
Carriers:         19 | On Land:          3 | In Stream:         16
Carriers:         14 | On Land:          3 | In Stream:         11
Carriers:          9 | On Land:          2 | In Stream:          7
Carriers:          4 | On Land:          2 | In Stream:          2
Carriers:          2 | On Land:          2 | In Stream:          0
Carriers:          2 | On Land:          2 | In Stream:          0
Carriers:          1 | On Land:          1 | In Stream:          0
Carriers:          1 | On Land:          1 | In Stream:          0
Carriers:          1 | On Land:          1 | In Stream:          0
Carriers:          1 | On Land:          1 | In Stream:          0
Carriers:          1 | On Land:          1 | In Stream:          0
Carriers:          1 | On Land:          1 | In Stream:          0
All done.
Simulation complete.
Users affected: 19
Returning the results map to Ferd's code.
"Elapsed time: 18269.246086 msecs"
blocked-use being stored as an observation of 
soilretentionEcology:LostValuableSediment
Resampling matrix from 39 x 85 to 118 x 256...
actual-use being stored as an observation of 
soilretentionEcology:ActualSedimentDepositionBeneficiaries
  Distinct Layer Values: [Pre] 16 [Post] 106
Resampling matrix from 39 x 85 to 118 x 256...
possible-use being stored as an observation of 
soilretentionEcology:PossibleSedimentDepositionBeneficiaries
  Distinct Layer Values: [Pre] 20 [Post] 128
Resampling matrix from 39 x 85 to 118 x 256...
inaccessible-use being stored as an observation of 
soilretentionEcology:InaccessibleSedimentDepositionBeneficiaries
  Distinct Layer Values: [Pre] 20 [Post] 128
Resampling matrix from 39 x 85 to 118 x 256...
actual-flow being stored as an observation of 
soilretentionEcology:ActualSedimentFlow
  Distinct Layer Values: [Pre] 21 [Post] 169
Resampling matrix from 39 x 85 to 118 x 256...
actual-source being stored as an observation of 
soilretentionEcology:ActualSedimentSource
  Distinct Layer Values: [Pre] 3015 [Post] 12488
Resampling matrix from 39 x 85 to 118 x 256...
blocked-flow being stored as an observation of 
soilretentionEcology:AbsorbedSedimentFlow
  Distinct Layer Values: [Pre] 42 [Post] 244
Resampling matrix from 39 x 85 to 118 x 256...
No observable defined for :inaccessible-sink: skipping
inaccessible-source being stored as an observation of 
soilretentionEcology:UnutilizedSedimentSource
  Distinct Layer Values: [Pre] 265 [Post] 1804
Resampling matrix from 39 x 85 to 118 x 256...
theoretical-use being stored as an observation of 
soilretentionEcology:PotentialSedimentDepositionBeneficiaries
  Distinct Layer Values: [Pre] 3148 [Post] 12524
Resampling matrix from 39 x 85 to 118 x 256...
possible-source being stored as an observation of 
soilretentionEcology:PossibleSedimentSource
  Distinct Layer Values: [Pre] 2 [Post] 111
Resampling matrix from 39 x 85 to 118 x 256...
blocked-source being stored as an observation of 
soilretentionEcology:NegatedSedimentSource
  Distinct Layer Values: [Pre] 42 [Post] 244
Resampling matrix from 39 x 85 to 118 x 256...
theoretical-source being stored as an observation of 
soilretentionEcology:MaximumSedimentSource
  Distinct Layer Values: [Pre] 38 [Post] 218
Resampling matrix from 39 x 85 to 118 x 256...
possible-flow being stored as an observation of 
soilretentionEcology:PossibleSedimentFlow
  Distinct Layer Values: [Pre] 3189 [Post] 12622
Resampling matrix from 39 x 85 to 118 x 256...
theoretical-sink being stored as an observation of 
soilretentionEcology:MaximumPotentialDeposition
  Distinct Layer Values: [Pre] 3011 [Post] 12484
Resampling matrix from 39 x 85 to 118 x 256...
actual-sink being stored as an observation of 
soilretentionEcology:UtilizedDeposition
  Distinct Layer Values: [Pre] 528 [Post] 3033
Resampling matrix from 39 x 85 to 118 x 256...
Spatially distributed on a 256 by 118 raster grid (cell area = 191,072.267 m²)
----------------------------------------------------------------------------
soilretentionEcology:PossibleSedimentFlow
----------------------------------------------------------------------------
[0 808,659.428]               |***********************************|   29227
[808,659.428 1,617,318.856]   |                                   |     732
[1,617,318.856 2,425,978.284] |                                   |     150
[2,425,978.284 3,234,637.713] |                                   |      54
[3,234,637.713 4,043,297.141] |                                   |      22
[4,043,297.141 4,851,956.569] |                                   |       6
[4,851,956.569 5,660,615.997] |                                   |       9
[5,660,615.997 6,469,275.425] |                                   |       2
[6,469,275.425 7,277,934.853] |                                   |       2
[7,277,934.853 8,086,594.281] |                                   |       4
----------------------------------------------------------------------------
no-data                       |                                   |       0

----------------------------------------------------------------------------
soilretentionEcology:FloodplainWidthClass
----------------------------------------------------------------------------
soilretentionEcology:VeryNarrowFloodplain |                       |     887
soilretentionEcology:NarrowFloodplain     |                       |     258
soilretentionEcology:VeryWideFloodplain   |                       |      80
----------------------------------------------------------------------------
no-data                                   |***********************|   28983

----------------------------------------------------------------------------
soilretentionEcology:LostValuableSediment
----------------------------------------------------------------------------
[0 34,150.295]            |***************************************|   30172
[34,150.295 68,300.59]    |                                       |      24
[68,300.59 102,450.885]   |                                       |       3
[102,450.885 136,601.181] |                                       |       0
[136,601.181 170,751.476] |                                       |       0
[170,751.476 204,901.771] |                                       |       0
[204,901.771 239,052.066] |                                       |       3
[239,052.066 273,202.361] |                                       |       0
[273,202.361 307,352.656] |                                       |       2
[307,352.656 341,502.951] |                                       |       4
----------------------------------------------------------------------------
no-data                   |                                       |       0

----------------------------------------------------------------------------
soilretentionEcology:InaccessibleSedimentDepositionBeneficiaries
----------------------------------------------------------------------------
[0 15,213,308.503]                |*******************************|   29856
[15,213,308.503 30,426,617.006]   |                               |      23
[30,426,617.006 45,639,925.509]   |                               |      30
[45,639,925.509 60,853,234.012]   |                               |      15
[60,853,234.012 76,066,542.515]   |                               |       7
[76,066,542.515 91,279,851.018]   |                               |      19
[91,279,851.018 106,493,159.521]  |                               |      19
[106,493,159.521 121,706,468.024] |                               |      23
[121,706,468.024 136,919,776.527] |                               |     211
[136,919,776.527 152,133,085.03]  |                               |       5
----------------------------------------------------------------------------
no-data                           |                               |       0

----------------------------------------------------------------------------
soilretentionEcology:DepositionProneFarmers
----------------------------------------------------------------------------
[0 0.1]   |*******************************************************|   30186
[0.1 0.2] |                                                       |       0
[0.2 0.3] |                                                       |       0
[0.3 0.4] |                                                       |       0
[0.4 0.5] |                                                       |       0
[0.5 0.6] |                                                       |       0
[0.6 0.7] |                                                       |       0
[0.7 0.8] |                                                       |       0
[0.8 0.9] |                                                       |       0
[0.9 1]   |                                                       |      22
----------------------------------------------------------------------------
no-data   |                                                       |       0

----------------------------------------------------------------------------
infrastructure:Levee
----------------------------------------------------------------------------
[1 1]   |*                                                        |     572
[1 1]   |                                                         |       0
[1 1]   |                                                         |       0
[1 1]   |                                                         |       0
[1 1]   |                                                         |       0
[1 1]   |                                                         |       0
[1 1]   |                                                         |       0
[1 1]   |                                                         |       0
[1 1]   |                                                         |       0
[1 1]   |                                                         |       0
----------------------------------------------------------------------------
no-data |*********************************************************|   29636

----------------------------------------------------------------------------
soilretentionEcology:HydrologicSoilsGroup
----------------------------------------------------------------------------
soilretentionEcology:SoilGroupA |**********                       |    2758
soilretentionEcology:SoilGroupC |*********************************|    9088
soilretentionEcology:SoilGroupD |*********                        |    2692
soilretentionEcology:SoilGroupB |*****************************    |    8190
----------------------------------------------------------------------------
no-data                         |***************************      |    7480

----------------------------------------------------------------------------
soilretentionEcology:MaximumPotentialDeposition
----------------------------------------------------------------------------
[0 2,500]       |*************************************************|   27423
[2,500 5,000]   |**                                               |    1295
[5,000 7,500]   |*                                                |     603
[7,500 10,000]  |                                                 |     292
[10,000 12,500] |                                                 |     178
[12,500 15,000] |                                                 |     118
[15,000 17,500] |                                                 |     103
[17,500 20,000] |                                                 |     106
[20,000 22,500] |                                                 |      86
[22,500 25,000] |                                                 |       4
----------------------------------------------------------------------------
no-data         |                                                 |       0

----------------------------------------------------------------------------
soilretentionEcology:UtilizedDeposition
----------------------------------------------------------------------------
[0 486.626]           |*******************************************|   30160
[486.626 973.251]     |                                           |      18
[973.251 1,459.877]   |                                           |       9
[1,459.877 1,946.502] |                                           |      12
[1,946.502 2,433.128] |                                           |       0
[2,433.128 2,919.753] |                                           |       1
[2,919.753 3,406.379] |                                           |       2
[3,406.379 3,893.004] |                                           |       0
[3,893.004 4,379.63]  |                                           |       2
[4,379.63 4,866.255]  |                                           |       4
----------------------------------------------------------------------------
no-data               |                                           |       0

----------------------------------------------------------------------------
soilretentionEcology:UnutilizedSedimentSource
----------------------------------------------------------------------------
[0 16,763.172]            |***************************************|    8140
[16,763.172 33,526.344]   |*****************************          |    6211
[33,526.344 50,289.516]   |********************                   |    4294
[50,289.516 67,052.688]   |************************               |    5171
[67,052.688 83,815.86]    |*******************                    |    4022
[83,815.86 100,579.031]   |*******                                |    1531
[100,579.031 117,342.203] |***                                    |     651
[117,342.203 134,105.375] |                                       |     169
[134,105.375 150,868.547] |                                       |      15
[150,868.547 167,631.719] |                                       |       4
----------------------------------------------------------------------------
no-data                   |                                       |       0

----------------------------------------------------------------------------
soilretentionEcology:SlopeStabilityClass
----------------------------------------------------------------------------
soilretentionEcology:HighSlopeStability     |*********************|   19622
soilretentionEcology:LowSlopeStability      |***                  |    3172
soilretentionEcology:ModerateSlopeStability |****                 |    4262
----------------------------------------------------------------------------
no-data                                     |***                  |    3152

----------------------------------------------------------------------------
soilretentionEcology:SedimentSourceValueAnnual
----------------------------------------------------------------------------
[m=775.005]     |                                                 |       2
[m=775.005]     |                                                 |       2
[m=791.255]     |                                                 |       1
[m=791.255]     |                                                 |       4
[m=791.255]     |                                                 |       1
[m=922.505]     |                                                 |       2
[m=922.505]     |                                                 |       3
[m=922.505]     |                                                 |       9
[m=925.005]     |                                                 |       2
[m=957.505]     |                                                 |       1
[m=957.505]     |                                                 |       6
[m=957.505]     |                                                 |       2
[m=988.755]     |                                                 |       1
[m=1,022.505]   |                                                 |       1
[m=1,022.505]   |                                                 |       2
[m=1,053.755]   |                                                 |       1
[m=1,053.755]   |                                                 |       2
[m=1,150.005]   |                                                 |       2
[m=1,150.005]   |                                                 |      22
[m=1,150.005]   |                                                 |       4
[m=1,212.505]   |                                                 |       1
[m=1,212.505]   |                                                 |       4
[m=1,250.005]   |                                                 |       1
[m=1,250.005]   |                                                 |       3
[m=1,282.505]   |                                                 |       5
[m=1,282.505]   |                                                 |       8
[m=1,342.505]   |                                                 |       6
[m=1,342.505]   |                                                 |       1
[m=1,347.505]   |                                                 |       1
[m=1,475.005]   |                                                 |       2
[m=1,475.005]   |                                                 |      10
[m=1,475.005]   |                                                 |       8
[m=1,565.005]   |                                                 |       5
[m=1,565.005]   |                                                 |       4
[m=1,565.005]   |                                                 |       1
[m=1,565.005]   |                                                 |       1
[m=1,565.005]   |                                                 |       1
[m=1,647.505]   |                                                 |       3
[m=1,647.505]   |                                                 |       2
[m=1,667.505]   |                                                 |       1
[m=1,878.755]   |                                                 |       2
[m=1,945.005]   |                                                 |       1
[m=1,945.005]   |                                                 |       2
[m=1,945.005]   |                                                 |       1
[m=1,945.005]   |                                                 |      10
[m=1,945.005]   |                                                 |       1
[m=1,945.005]   |                                                 |       7
[m=1,945.005]   |                                                 |       3
[m=1,945.005]   |                                                 |      12
[m=2,040.005]   |                                                 |       6
[m=2,040.005]   |                                                 |      11
[m=2,093.755]   |                                                 |       1
[m=2,160.005]   |                                                 |       1
[m=2,160.005]   |                                                 |       2
[m=2,160.005]   |                                                 |       2
[m=2,207.505]   |                                                 |       5
[m=2,207.505]   |                                                 |       2
[m=2,207.505]   |                                                 |       4
[m=2,270.005]   |                                                 |       2
[m=2,270.005]   |                                                 |       2
[m=2,270.005]   |                                                 |       5
[m=2,270.005]   |                                                 |       2
[m=2,270.005]   |                                                 |       1
[m=2,292.505]   |                                                 |       2
[m=2,292.505]   |                                                 |       2
[m=2,358.755]   |                                                 |       2
[m=2,365.005]   |                                                 |       6
[m=2,365.005]   |                                                 |       5
[m=2,425.005]   |                                                 |       1
[m=2,425.005]   |                                                 |       1
[m=2,491.255]   |                                                 |       2
[m=2,491.255]   |                                                 |       4
[m=2,512.505]   |                                                 |       4
[m=2,512.505]   |                                                 |       3
[m=2,537.505]   |                                                 |      13
[m=2,537.505]   |*                                                |      49
[m=2,537.505]   |                                                 |       4
[m=2,537.505]   |                                                 |       1
[m=2,570.005]   |                                                 |       9
[m=2,570.005]   |                                                 |      14
[m=2,570.005]   |                                                 |       2
[m=2,570.005]   |                                                 |       1
[m=2,570.005]   |*                                                |      59
[m=2,575.005]   |                                                 |      10
[m=2,575.005]   |                                                 |       6
[m=2,610.005]   |                                                 |       4
[m=2,610.005]   |                                                 |      12
[m=2,617.505]   |                                                 |       2
[m=2,635.005]   |                                                 |       1
[m=2,635.005]   |                                                 |       5
[m=2,635.005]   |                                                 |      30
[m=2,635.005]   |                                                 |       9
[m=2,635.005]   |*                                                |      80
[m=2,652.505]   |                                                 |       7
[m=2,652.505]   |                                                 |      10
[m=2,700.005]   |                                                 |       2
[m=2,700.005]   |                                                 |       1
[m=2,717.505]   |                                                 |       4
[m=2,717.505]   |                                                 |       1
[m=2,717.505]   |                                                 |      30
[m=2,717.505]   |                                                 |      29
[m=2,717.505]   |                                                 |       1
[m=2,717.505]   |*                                                |      55
[m=2,717.505]   |*                                                |      53
[m=2,717.505]   |                                                 |       2
[m=2,750.005]   |                                                 |       3
[m=2,750.005]   |*                                                |      57
[m=2,750.005]   |******                                           |     288
[m=2,750.005]   |                                                 |      34
[m=2,750.005]   |                                                 |       3
[m=2,750.005]   |**************                                   |     653
[m=2,762.505]   |                                                 |       3
[m=2,822.505]   |                                                 |       3
[m=2,822.505]   |                                                 |       2
[m=2,837.505]   |                                                 |       1
[m=2,888.755]   |                                                 |       1
[m=2,900.005]   |                                                 |       9
[m=2,900.005]   |                                                 |       4
[m=2,941.255]   |                                                 |      16
[m=2,941.255]   |                                                 |      41
[m=2,941.255]   |                                                 |       4
[m=2,973.755]   |                                                 |       6
[m=2,973.755]   |                                                 |      24
[m=2,973.755]   |                                                 |      15
[m=2,973.755]   |                                                 |       7
[m=3,025.005]   |                                                 |       1
[m=3,038.755]   |                                                 |       4
[m=3,038.755]   |                                                 |      25
[m=3,038.755]   |                                                 |      13
[m=3,038.755]   |                                                 |       5
[m=3,038.755]   |                                                 |       2
[m=3,052.505]   |                                                 |       3
[m=3,052.505]   |                                                 |      24
[m=3,087.505]   |                                                 |       2
[m=3,117.505]   |                                                 |       5
[m=3,117.505]   |                                                 |      10
[m=3,117.505]   |*                                                |      48
[m=3,117.505]   |                                                 |      31
[m=3,117.505]   |**                                               |     129
[m=3,117.505]   |*                                                |      51
[m=3,133.755]   |                                                 |       2
[m=3,133.755]   |                                                 |       1
[m=3,150.005]   |                                                 |       4
[m=3,150.005]   |**                                               |     123
[m=3,150.005]   |                                                 |       2
[m=3,150.005]   |*                                                |      68
[m=3,305.005]   |                                                 |       1
[m=3,305.005]   |                                                 |       1
[m=3,417.967]   |                                                 |       1
[m=3,450.38]    |                                                 |       1
[m=3,594.78]    |                                                 |       2
[m=3,702.255]   |                                                 |       4
[m=3,775.005]   |                                                 |      12
[m=3,775.005]   |                                                 |      15
[m=3,786.867]   |                                                 |       1
[m=3,813.592]   |                                                 |      15
[m=3,900.005]   |                                                 |       2
[m=3,900.005]   |                                                 |       3
[m=3,900.005]   |                                                 |       1
[m=3,900.005]   |                                                 |       1
[m=4,015.005]   |                                                 |       1
[m=4,015.005]   |                                                 |       1
[m=4,015.005]   |                                                 |       1
[m=4,025.005]   |                                                 |       1
[m=4,025.005]   |                                                 |       2
[m=4,036.267]   |                                                 |       1
[m=4,056.03]    |                                                 |       4
[m=4,147.505]   |                                                 |       4
[m=4,150.005]   |                                                 |       1
[m=4,150.005]   |                                                 |       1
[m=4,217.467]   |                                                 |       1
[m=4,275.005]   |                                                 |       1
[m=4,275.005]   |                                                 |       3
[m=4,275.005]   |                                                 |       6
[m=4,275.005]   |                                                 |       1
[m=4,275.005]   |                                                 |       1
[m=4,275.005]   |                                                 |       6
[m=4,281.242]   |                                                 |       1
[m=4,337.505]   |                                                 |       1
[m=4,400.005]   |                                                 |       1
[m=4,400.005]   |                                                 |       1
[m=4,400.005]   |                                                 |       2
[m=4,448.255]   |                                                 |       6
[m=4,448.255]   |                                                 |       2
[m=4,517.28]    |                                                 |      36
[m=4,598.43]    |                                                 |      12
[m=4,775.617]   |                                                 |       7
[m=4,810.005]   |                                                 |       3
[m=4,839.755]   |                                                 |       3
[m=4,898.78]    |                                                 |       5
[m=4,910.755]   |                                                 |      14
[m=5,024.13]    |                                                 |       1
[m=5,077.18]    |                                                 |      27
[m=5,207.505]   |                                                 |       1
[m=5,220.967]   |                                                 |       6
[m=5,410.03]    |                                                 |       2
[m=5,414.755]   |                                                 |      30
[m=5,559.83]    |                                                 |       2
[m=5,630.38]    |                                                 |       3
[m=5,737.505]   |                                                 |       2
[m=5,870.005]   |                                                 |       2
[m=5,985.355]   |                                                 |       1
[m=5,989.755]   |**                                               |     106
[m=6,012.505]   |                                                 |      11
[m=6,156.43]    |                                                 |      27
[m=6,231.08]    |                                                 |       2
[m=6,231.505]   |                                                 |       6
[m=6,236.63]    |                                                 |      10
[m=6,267.505]   |                                                 |       1
[m=6,267.505]   |                                                 |       1
[m=6,400.005]   |                                                 |       1
[m=6,400.005]   |                                                 |       1
[m=6,444.28]    |                                                 |       1
[m=6,637.505]   |                                                 |      25
[m=6,767.855]   |                                                 |       3
[m=6,767.855]   |                                                 |       3
[m=6,797.68]    |                                                 |      42
[m=6,902.33]    |                                                 |      19
[m=6,947.83]    |                                                 |       2
[m=6,969.005]   |*                                                |      53
[m=7,096.28]    |                                                 |       5
[m=7,118.03]    |                                                 |       1
[m=7,205.88]    |                                                 |       3
[m=7,550.355]   |                                                 |      14
[m=7,550.355]   |                                                 |      11
[m=7,550.355]   |                                                 |       8
[m=7,603.555]   |                                                 |       1
[m=7,706.505]   |*                                                |      53
[m=7,781.58]    |                                                 |       2
[m=7,852.255]   |                                                 |       1
[m=7,902.405]   |                                                 |       1
[m=7,920.03]    |                                                 |      17
[m=7,921.855]   |                                                 |       2
[m=7,974.63]    |                                                 |       5
[m=7,978.342]   |                                                 |       4
[m=8,108.592]   |                                                 |       2
[m=8,239.755]   |                                                 |       4
[m=8,504.217]   |                                                 |       9
[m=8,576.055]   |                                                 |      26
[m=8,576.055]   |                                                 |       6
[m=8,576.055]   |                                                 |       5
[m=8,615.33]    |                                                 |       2
[m=8,757.192]   |                                                 |       4
[m=8,917.717]   |                                                 |       3
[m=8,941.855]   |                                                 |      21
[m=8,982.405]   |                                                 |       3
[m=8,995.755]   |                                                 |       2
[m=9,075.305]   |                                                 |       2
[m=9,075.305]   |                                                 |       1
[m=9,075.305]   |                                                 |       1
[m=9,084.63]    |                                                 |       4
[m=9,114.155]   |                                                 |       2
[m=9,159.68]    |                                                 |       4
[m=9,159.68]    |                                                 |       1
[m=9,194.805]   |                                                 |       1
[m=9,194.805]   |                                                 |       6
[m=9,204.092]   |                                                 |       2
[m=9,408.605]   |                                                 |       2
[m=9,415.155]   |                                                 |      15
[m=9,429.88]    |                                                 |       2
[m=9,432.83]    |                                                 |       1
[m=9,441.98]    |                                                 |       1
[m=9,548.555]   |*                                                |      55
[m=9,548.555]   |                                                 |      24
[m=9,575.405]   |                                                 |       4
[m=9,614.255]   |                                                 |       2
[m=9,662.33]    |                                                 |       3
[m=9,662.33]    |                                                 |       3
[m=9,772.505]   |                                                 |       1
[m=9,772.505]   |*                                                |      63
[m=9,804.067]   |                                                 |      11
[m=9,889.08]    |                                                 |       6
[m=9,961.855]   |*                                                |      74
[m=9,976.655]   |                                                 |       1
[m=9,983.005]   |                                                 |       1
[m=10,036.655]  |                                                 |      35
[m=10,062.405]  |                                                 |      19
[m=10,208.255]  |                                                 |       2
[m=10,210.305]  |                                                 |       6
[m=10,210.305]  |                                                 |      14
[m=10,210.305]  |                                                 |       3
[m=10,263.43]   |                                                 |       8
[m=10,353.38]   |                                                 |      30
[m=10,449.717]  |                                                 |       8
[m=10,479.117]  |                                                 |       1
[m=10,479.805]  |                                                 |      42
[m=10,479.805]  |                                                 |      11
[m=10,511.242]  |                                                 |       4
[m=10,579.08]   |                                                 |       2
[m=10,591.105]  |                                                 |      34
[m=10,639.055]  |                                                 |       2
[m=10,686.13]   |                                                 |      13
[m=10,732.655]  |*                                                |      53
[m=10,732.655]  |                                                 |       6
[m=10,768.23]   |                                                 |      13
[m=10,773.255]  |                                                 |       1
[m=10,773.255]  |                                                 |      11
[m=10,837.417]  |                                                 |       7
[m=10,901.755]  |                                                 |      33
[m=11,021.08]   |*                                                |      49
[m=11,021.08]   |                                                 |      32
[m=11,021.08]   |                                                 |       2
[m=11,025.38]   |                                                 |       1
[m=11,035.33]   |                                                 |      14
[m=11,042.217]  |                                                 |       1
[m=11,103.342]  |                                                 |      14
[m=11,117.805]  |                                                 |      13
[m=11,147.505]  |                                                 |      14
[m=11,147.505]  |*****                                            |     224
[m=11,235.755]  |                                                 |       5
[m=11,235.755]  |                                                 |      18
[m=11,273.517]  |                                                 |       3
[m=11,273.517]  |                                                 |       4
[m=11,281.655]  |                                                 |       7
[m=11,281.655]  |                                                 |       3
[m=11,333.005]  |                                                 |       3
[m=11,345.304]  |                                                 |       9
[m=11,345.304]  |                                                 |      19
[m=11,345.304]  |                                                 |      11
[m=11,420.755]  |                                                 |       2
[m=11,420.755]  |                                                 |      10
[m=11,542.979]  |                                                 |       3
[m=11,548.629]  |                                                 |       6
[m=11,569.092]  |                                                 |       3
[m=11,589.579]  |                                                 |       4
[m=11,596.555]  |                                                 |      29
[m=11,613.867]  |                                                 |       2
[m=11,613.867]  |                                                 |       1
[m=11,622.13]   |***                                              |     134
[m=11,628.754]  |                                                 |       1
[m=11,642.254]  |                                                 |       1
[m=11,669.53]   |                                                 |      14
[m=11,695.342]  |                                                 |      15
[m=11,764.804]  |****                                             |     207
[m=11,764.804]  |                                                 |      44
[m=11,773.604]  |                                                 |      32
[m=11,806.905]  |                                                 |       3
[m=11,883.329]  |                                                 |      13
[m=11,942.379]  |                                                 |      14
[m=12,007.967]  |                                                 |       2
[m=12,050.154]  |*******                                          |     311
[m=12,053.504]  |                                                 |       1
[m=12,053.504]  |                                                 |       5
[m=12,094.479]  |*                                                |      45
[m=12,189.254]  |*                                                |      49
[m=12,244.517]  |                                                 |       1
[m=12,318.155]  |                                                 |       4
[m=12,379.829]  |*                                                |      49
[m=12,379.829]  |**                                               |     124
[m=12,379.829]  |                                                 |       8
[m=12,403.592]  |                                                 |       2
[m=12,411.63]   |                                                 |       5
[m=12,444.129]  |                                                 |       4
[m=12,451.167]  |                                                 |       3
[m=12,522.504]  |****************************                     |    1277
[m=12,522.504]  |**                                               |      97
[m=12,573.967]  |*                                                |      77
[m=12,586.654]  |                                                 |      22
[m=12,681.004]  |                                                 |      39
[m=12,683.004]  |                                                 |       8
[m=12,760.392]  |                                                 |       8
[m=12,760.392]  |                                                 |      14
[m=12,898.329]  |                                                 |       2
[m=12,929.879]  |                                                 |      11
[m=12,986.629]  |*                                                |      46
[m=13,069.229]  |                                                 |      12
[m=13,080.779]  |                                                 |      27
[m=13,133.242]  |                                                 |      12
[m=13,133.242]  |                                                 |       6
[m=13,154.754]  |                                                 |       2
[m=13,367.867]  |                                                 |       3
[m=13,391.629]  |                                                 |       3
[m=13,442.079]  |*                                                |      87
[m=13,442.079]  |*                                                |      57
[m=13,477.254]  |                                                 |       3
[m=13,487.504]  |                                                 |       3
[m=13,535.292]  |                                                 |       1
[m=13,561.629]  |**                                               |      97
[m=13,628.504]  |                                                 |       5
[m=13,628.504]  |*                                                |      72
[m=13,826.429]  |                                                 |       2
[m=13,862.879]  |                                                 |       5
[m=13,999.754]  |                                                 |       3
[m=14,044.592]  |*****                                            |     249
[m=14,083.504]  |                                                 |       4
[m=14,095.854]  |                                                 |       1
[m=14,112.504]  |                                                 |      18
[m=14,112.504]  |                                                 |       3
[m=14,115.204]  |                                                 |       1
[m=14,131.004]  |*                                                |      53
[m=14,184.204]  |                                                 |       1
[m=14,247.267]  |                                                 |      39
[m=14,247.267]  |***                                              |     135
[m=14,466.217]  |                                                 |       2
[m=14,467.679]  |*                                                |      50
[m=14,595.479]  |*                                                |      63
[m=14,633.529]  |                                                 |       4
[m=14,652.617]  |**                                               |      97
[m=14,652.617]  |                                                 |      25
[m=14,667.254]  |                                                 |       2
[m=14,702.854]  |                                                 |       1
[m=14,737.504]  |                                                 |      11
[m=14,737.504]  |**                                               |     111
[m=14,792.254]  |                                                 |       1
[m=14,855.454]  |                                                 |       3
[m=14,935.379]  |                                                 |       1
[m=15,000.829]  |**                                               |     116
[m=15,000.829]  |**                                               |     118
[m=15,000.829]  |                                                 |      20
[m=15,102.167]  |                                                 |       1
[m=15,108.929]  |***                                              |     138
[m=15,178.029]  |                                                 |       2
[m=15,203.504]  |*****                                            |     223
[m=15,203.504]  |*                                                |      58
[m=15,359.404]  |                                                 |       1
[m=15,485.354]  |                                                 |       3
[m=15,485.354]  |                                                 |       1
[m=15,526.704]  |                                                 |      16
[m=15,574.754]  |                                                 |       1
[m=15,682.129]  |                                                 |       5
[m=15,767.704]  |                                                 |       5
[m=15,767.704]  |                                                 |       2
[m=15,851.779]  |                                                 |      14
[m=15,887.254]  |*                                                |      56
[m=16,103.354]  |                                                 |       1
[m=16,103.354]  |                                                 |       1
[m=16,173.254]  |                                                 |       4
[m=16,183.154]  |                                                 |       3
[m=16,267.854]  |                                                 |      15
[m=16,267.854]  |                                                 |       4
[m=16,267.854]  |                                                 |       8
[m=16,341.004]  |                                                 |       6
[m=16,341.004]  |                                                 |       4
[m=16,419.629]  |**                                               |     108
[m=16,598.779]  |                                                 |       3
[m=16,635.754]  |                                                 |       1
[m=16,635.754]  |                                                 |      14
[m=16,679.679]  |                                                 |       9
[m=16,804.754]  |                                                 |       9
[m=16,950.879]  |                                                 |       4
[m=17,006.904]  |                                                 |      15
[m=17,157.129]  |****                                             |     205
[m=17,158.429]  |                                                 |      11
[m=17,248.504]  |                                                 |       3
[m=17,712.254]  |                                                 |      10
[m=17,712.254]  |                                                 |       7
[m=17,712.254]  |                                                 |      19
[m=17,719.629]  |                                                 |      20
[m=17,774.754]  |*                                                |      80
[m=18,110.854]  |                                                 |       4
[m=18,110.854]  |                                                 |       3
[m=18,110.854]  |                                                 |       3
[m=18,166.004]  |                                                 |       6
[m=18,166.004]  |                                                 |       7
[m=18,203.779]  |                                                 |       1
[m=18,291.004]  |                                                 |      12
[m=18,291.004]  |                                                 |      15
[m=18,371.579]  |                                                 |       4
[m=18,672.529]  |                                                 |      15
[m=18,817.254]  |                                                 |       8
[m=18,959.592]  |                                                 |       4
[m=19,100.904]  |                                                 |       4
[m=19,205.329]  |                                                 |      11
[m=19,238.554]  |                                                 |       2
[m=19,238.554]  |                                                 |       1
[m=19,323.504]  |                                                 |       2
[m=19,323.504]  |                                                 |       6
[m=19,408.504]  |                                                 |       5
[m=19,849.754]  |                                                 |      13
[m=19,849.754]  |                                                 |      27
[m=19,849.754]  |                                                 |       8
[m=19,983.504]  |                                                 |      29
[m=20,026.229]  |                                                 |       3
[m=20,042.404]  |                                                 |       4
[m=20,082.967]  |                                                 |       1
[m=20,107.879]  |                                                 |       1
[m=20,111.629]  |                                                 |       1
[m=20,211.054]  |                                                 |      24
[m=20,211.054]  |                                                 |      18
[m=20,211.054]  |                                                 |      10
[m=20,337.504]  |                                                 |       6
[m=20,366.004]  |                                                 |       4
[m=20,366.004]  |                                                 |      17
[m=20,443.479]  |                                                 |       1
[m=20,647.817]  |                                                 |       6
[m=20,717.879]  |                                                 |       3
[m=20,962.504]  |                                                 |      23
[m=20,962.504]  |                                                 |       1
[m=21,045.754]  |                                                 |       5
[m=21,045.754]  |                                                 |       3
[m=21,046.229]  |*                                                |      60
[m=21,110.304]  |                                                 |       1
[m=21,122.404]  |                                                 |      13
[m=21,183.554]  |                                                 |      16
[m=21,183.554]  |                                                 |      38
[m=21,183.554]  |                                                 |      30
[m=21,218.817]  |                                                 |       1
[m=21,496.429]  |                                                 |      19
[m=21,505.904]  |                                                 |      10
[m=21,532.842]  |                                                 |       6
[m=21,587.504]  |                                                 |       6
[m=21,587.504]  |***                                              |     135
[m=21,671.004]  |                                                 |       1
[m=22,066.229]  |*                                                |      74
[m=22,071.579]  |                                                 |       4
[m=22,080.079]  |                                                 |       1
[m=22,137.329]  |                                                 |       2
[m=22,137.679]  |**                                               |     123
[m=22,214.054]  |                                                 |       5
[m=22,214.054]  |                                                 |       2
[m=22,258.254]  |                                                 |      14
[m=22,367.167]  |                                                 |       4
[m=22,477.629]  |                                                 |       5
[m=22,525.879]  |                                                 |       3
[m=22,546.167]  |                                                 |       6
[m=22,778.467]  |                                                 |      40
[m=22,808.579]  |                                                 |       3
[m=22,910.304]  |                                                 |       2
[m=22,910.304]  |                                                 |       3
[m=22,951.054]  |                                                 |       6
[m=22,951.054]  |                                                 |       1
[m=22,953.054]  |                                                 |       2
[m=22,953.054]  |                                                 |       1
[m=23,110.929]  |                                                 |       1
[m=23,110.929]  |                                                 |       1
[m=23,207.204]  |                                                 |       3
[m=23,238.029]  |                                                 |      11
[m=23,420.354]  |                                                 |       2
[m=23,444.729]  |                                                 |       1
[m=23,470.754]  |*                                                |      85
[m=23,470.754]  |                                                 |      12
[m=23,571.979]  |                                                 |       1
[m=23,657.754]  |                                                 |       4
[m=23,749.229]  |                                                 |       3
[m=23,767.379]  |                                                 |       1
[m=23,794.629]  |                                                 |      25
[m=23,801.404]  |                                                 |       8
[m=23,818.892]  |                                                 |       1
[m=23,903.904]  |                                                 |       1
[m=24,010.154]  |                                                 |       3
[m=24,024.092]  |**                                               |     104
[m=24,045.304]  |                                                 |      30
[m=24,045.304]  |                                                 |      14
[m=24,045.304]  |                                                 |      12
[m=24,226.754]  |                                                 |       2
[m=24,236.054]  |                                                 |      26
[m=24,236.054]  |                                                 |       2
[m=24,353.454]  |                                                 |       8
[m=24,395.254]  |                                                 |      39
[m=24,395.579]  |                                                 |       2
[m=24,422.329]  |                                                 |       4
[m=24,446.279]  |                                                 |       2
[m=24,539.117]  |                                                 |       2
[m=24,847.504]  |                                                 |      36
[m=24,847.504]  |                                                 |       2
[m=24,898.229]  |                                                 |       5
[m=24,931.729]  |*                                                |      51
[m=25,023.629]  |                                                 |       8
[m=25,063.379]  |**                                               |     129
[m=25,118.904]  |                                                 |      18
[m=25,118.904]  |                                                 |       1
[m=25,130.767]  |                                                 |      14
[m=25,132.754]  |*****                                            |     221
[m=25,180.304]  |                                                 |      31
[m=25,180.304]  |                                                 |      40
[m=25,180.304]  |*                                                |      65
[m=25,346.379]  |                                                 |      11
[m=25,362.416]  |                                                 |       1
[m=25,421.404]  |*                                                |      50
[m=25,514.254]  |*                                                |      45
[m=25,521.054]  |                                                 |      27
[m=25,521.054]  |***                                              |     152
[m=25,781.079]  |                                                 |       1
[m=25,781.079]  |                                                 |       9
[m=25,781.079]  |                                                 |      14
[m=25,888.054]  |                                                 |       6
[m=25,888.054]  |                                                 |       4
[m=25,927.129]  |                                                 |      28
[m=25,983.491]  |                                                 |       2
[m=26,114.229]  |***                                              |     134
[m=26,222.504]  |                                                 |       6
[m=26,222.504]  |*                                                |      67
[m=26,224.479]  |                                                 |      36
[m=26,279.879]  |                                                 |      13
[m=26,436.404]  |***                                              |     142
[m=26,436.404]  |                                                 |      12
[m=26,445.979]  |                                                 |       1
[m=26,487.466]  |                                                 |       1
[m=26,495.504]  |                                                 |       4
[m=26,694.779]  |                                                 |       7
[m=26,694.779]  |

kbagstad commented 13 years ago

Ferd, remember to do model -o, not model -d, as -d seems to consistently produce interesting results (per Gary's comment above) while -o is what's mysteriously leaving us with zeroes.

lambdatronic commented 13 years ago

Okay, so I need to report a related problem that has been hitting me repeatedly over the past few days.

Whenever I run any of the sediment or flood models (not sure if it happens more frequently with -o than with -d), I am seeing some non-deterministic behavior from ARIES. Specifically, I am passing the floodplains-code model (observable concept = soilretentionEcology:Floodplains) in the :context list of each of the sediment span statements, but when this observation is passed to my SPAN code, the unpacker sometimes gets a binary-coding and sometimes gets a classification. Because my code only accepts numeric-valued observations, it crashes when it hits a classification (because it doesn't know how to handle a Concept object). The weird thing is that sometimes when I run the models, they get the binary-coding and everything works fine. Other times (with no changes to anything), I'll get the classification, and everything crashes. What the heck?

> model -o foo.nc core.models.sediment-puget/sediment-farmers core.contexts.beta/wria9_256

... lots of INFO statements and then some SPAN messages...

Extracting flow layers: (Altitude Floodplains Levee River)
Inside unpack-datasource!
Checking datasource type... #<MemDoubleContextualizedDatasource MD[geophysics:Altitude: ]>
It's deterministic.
Inside unpack-datasource!
Checking datasource type... #<MemObjectContextualizedDatasource MO[soilretentionEcology:Floodplains: ]>
It's probabilistic and continuous.
Application start failed.
java.lang.RuntimeException: java.lang.IllegalArgumentException: No matching field found: getRanges for class org.integratedmodelling.thinklab.owlapi.Concept
        at clojure.lang.LazyCons.first(LazyCons.java:44)
        at clojure.lang.RT.first(RT.java:539)
        at clojure.core$first__2743.invoke(core.clj:40)
        at clojure.core$into__3590.invoke(core.clj:1620)
        at clj_span.aries_span_bridge$layer_map_from_observation__5891.invoke(aries_span_bridge.clj:262)
        at clj_span.aries_span_bridge$span_driver__5901.invoke(aries_span_bridge.clj:298)
        at aries$get_span_proxy__4050$fn__4052.invoke(aries-core.clj:25)
        at clojure.proxy.java.lang.Object$SPANProxy.runSPAN(Unknown Source)
        at org.integratedmodelling.aries.core.implementations.observations.SPANTransformer.transform(SPANTransformer.java:161)
        at org.integratedmodelling.corescience.context.ObservationContext.processTransformations(ObservationContext.java:719)
        at org.integratedmodelling.corescience.context.ObservationContext.processTransformations(ObservationContext.java:637)
        at org.integratedmodelling.corescience.context.ObservationContext.run(ObservationContext.java:758)
        at org.integratedmodelling.corescience.ObservationFactory.contextualize(ObservationFactory.java:73)
        at org.integratedmodelling.modelling.model.ModelFactory$ContextualizingModelResult.getResult(ModelFactory.java:253)
        at org.integratedmodelling.modelling.commands.ModelCommand.execute(ModelCommand.java:132)
        at org.integratedmodelling.thinklab.command.CommandManager.submitCommand(CommandManager.java:144)
        at org.integratedmodelling.thinklab.commandline.Shell.startConsole(Shell.java:114)
        at org.integratedmodelling.thinklab.shell.ShellApplication.startApplication(ShellApplication.java:130)
        at org.java.plugin.boot.Boot.boot(Boot.java:346)
        at org.java.plugin.boot.Boot.main(Boot.java:243)
Caused by: java.lang.IllegalArgumentException: No matching field found: getRanges for class org.integratedmodelling.thinklab.owlapi.Concept
        at clojure.lang.Reflector.getInstanceField(Reflector.java:235)
        at clojure.lang.Reflector.invokeNoArgInstanceMember(Reflector.java:257)
        at clj_span.aries_span_bridge$fn__5833.invoke(aries_span_bridge.clj:187)
        at clojure.lang.MultiFn.invoke(MultiFn.java:156)
        at clj_span.aries_span_bridge$layer_map_from_observation__5891$fn__5896.invoke(aries_span_bridge.clj:263)
        at clojure.core$map__3371$fn__3374.invoke(core.clj:1278)
        at clojure.lang.LazyCons.first(LazyCons.java:40)
        ... 19 more
Thinklab exited

So clearly that's not good, and it will get in your way while you are trying to test these models. Suggestions?

lambdatronic commented 13 years ago

And in response to Ken: Yes, GRASS sees zeros in the netCDF output layers as well, except for PossibleSedimentFlow, which has values (same as ARC and IDV). The problem is with the netCDFs.

fvilla commented 13 years ago

Thanks everyone - Gary, the only thing I can think of for the weirdness you're seeing is that there may be a model design error where the classification and the binary coding have the same concept - which is clearly a design error since the same concept should produce the same state. I do think such errors are caught but there may be a bug. In the case the model does that, the behavior may indeed be non-deterministic because the models are kept in maps and they may be added to the context in different orders - which has no effect if the models are kosher but could simply overwrite a state with another from an inner model if the concept is the same. I hope to have internet during my next workshop so I can check too.

That -d produces correct numbers is really puzzling in this case - there should be no classification involved and classifications are the only thing that needs to be unraveled if netcdfs are written. If I can try this out I'll see what I can do on Sunday.

lambdatronic commented 13 years ago

Good point on the fact that the classification and binary-coding have the same observable concept. Clearly not a good move on our part. The thing that really throws me there though is that I passed a specific model (floodplains-code) to the context list, so the fact that the system is actually grabbing a different model's states because it shares the same observable at floodplains-code is a bit scary to me.

The dangerous use case I see here is what happens if we make two models of the same concept which are of the same type (say both rankings, for example), but just compute the value differently. This behavior from the system makes me worry that I won't really know which model's states will be used when I run a model which depends on one of them.

Yikes!

And yeah, the netCDF writing thing is mysterious. Clearly, my models are running to completion successfully, but we're just losing the values when they get written out using the -o switch.

Good luck, boss.

fvilla commented 13 years ago

...but if you compute it differently, it's a different concept. States must be uniquely indexed by concepts - not just observation type, but also every detail of how it's computed. If it has the same concept it also must have the same numbers in the same context. The whole founding point of using a semantic system is that the concept identifies something completely and no ambiguity is allowed. Of course the system should tell you if you're breaking that model, which in this case it evidently does not. That's obviously a bug (such a check is in place but evidently there is something wrong here). The only difference allowed is when you use a scenario to provide an alternate model structure for one or more concepts, but the concept remains unique within the model being computed.

lambdatronic commented 13 years ago

Wait a moment. That doesn't sound right to me.

A fundamental element of this semantic system is the separation of observations and observables. When we construct a model with a defmodel statement, we are defining an observation of a particular observable, identified by its concept. However, if all observations of the same concept must have the same values, that means they are no longer separate entities but actually represent a one-to-one mapping.

If your claim is that observations may only have different values in different contexts, then this is no different than saying that an observable has different values in different contexts, as we still cannot separate observations and observables with this one-observable=one-observation constraint.

What happens if we wish to model wildfire risk first using a bayesian network and then using an arithmetic function of random variables? Both are computed observations of the same observable but represent different models of getting at the expected states of that observable.

fvilla commented 13 years ago

Aha. But that is only unique WITHIN a model - you can model the same concept any way you want as many times as you want (provided the observable concept is compatible with its observation model - if it's a classification it must classify to the same subclasses, if it's a boolean you can't make it a ranking, but that's obvious). But a model is an unambiguous statement of how you should observe a tree of interdependent observables, so you can't have the same concept mean different things within the same model - it's like having the same field name mean different things in a database query. One concept, one state within a result observation. Makes sense?

lambdatronic commented 13 years ago

Okay. I get the interdependent tree of variables thing, and that you can't have a classification and a binary-coding of the same observable (the mistake in the sediment and flood models). So what you are ultimately saying is that I can have any number of classifications of a classification observable, provided that they use the same subclasses, and that I can have any number of measurements of a measurement observable, provided that I use the same units, etc. Fair enough. Case closed.

But more importantly...how's the netCDF writing thing coming along? :-)

fvilla commented 13 years ago

Tried the model thing twice while discussing valuation of coastal services with 6 others, got zeros with -d, got puzzled, found that GS wasn't serving floodplain data so no users, restarted geoserver while still discussing, got use points but still zeros with -d, time over, now past midnight and still discussing valuation BS, tomorrow is travel all day. Will start again on the 7th hoping for better luck. So far my -d output is different from yours and I get the zeros anyway.... 'night f

lambdatronic commented 12 years ago

poke

kbagstad commented 12 years ago

Anything left to discuss here Ferd or Gary? If not can we close the issue?

lambdatronic commented 12 years ago

The flow model problem was caused (I believe) by the Floodplains layer not being passed correctly to the SPAN code. This should have been resolved when I ontologically separated the classification and binary-coding models into Floodplains and FloodplainsCode respectively. Now those models run fine AFAIK. Ferd will have to comment on the NetCDF writing. Was this resolved or not?

kbagstad commented 12 years ago

Fine as far as I know. Ferd, please close this unless there's anything going on that I'm not aware of.

ariesteam / aries

netcdf writing issues #14