USDAForestService / ForestVegetationSimulator

Forest Vegetation Simulation (FVS) - Growth and Yield Modeling software
Other
41 stars 26 forks source link

Inconsistent FVS outputs when running on the same OS with the same keywords on different days #37

Closed wuma2022 closed 8 months ago

wuma2022 commented 9 months ago

I grew one FIA plot (PLT_CN = 218220253020004) from SC using the keywords pasted below for FVS version 20230929 in both Windows 11 and Linux (Ubuntu v22.04.3 LTS).

StdIdent
04520141103305700068          SCJG__MX
StandCN
218220253020004
MgmtId
A001
InvYear         2014
TimeInt                      5
TimeInt            2         4
NumCycle          23

CYCLEAT   2023
CYCLEAT   2128
DATABASE
* FVS_Summary, FVS_Compute, Mistletoe
Summary            2
COMPUTDB           0         1
MisRpts            2
End

* FVS_TreeList,  FVS_Cutlist
Treelist           0                   0
Cutlist            0                   0
DATABASE
TREELIDB           2
CUTLIDB            2
End
*  FVS_Carbon, FVS_Hrv_Carbon, FVS_Fuels, FVS_Consumption
FMIn
CarbRept           2
CarbCut
CarbCalc           1         2    0.0425         9        11
FuelOut            0
FuelRept
End
DATABASE
CARBREDB           2
FUELREDB           2
FuelsOut           2
End

* FVS_SnagSum,  FVS_Down_Wood_Cov,  FVS_Down_Wood_Vol
FMIn
SnagSum
DWDVlout
DWDCvOut
End
DATABASE
SNAGSUDB           2
DWDVLDB            2
DWDCVDB            2
End

StrClass
DATABASE
STRCLSDB           2
End

DelOTab            1
DelOTab            2
DelOTab            4
!Exten:base Name:From: FVS_GroupAddFilesAndKeywords
DATABASE
StandSQL
SELECT * 
FROM FVS_StandInit_Plot
WHERE stand_cn = '%stand_cn%'
EndSQL
TreeSQL
SELECT * 
FROM FVS_TreeInit_Plot
WHERE stand_cn ='%stand_cn%'
EndSQL
END
!Exten:base Name:keyword.base.SDICalc
SDICalc            0         0         1
FMIn
!Exten:fire Name:keyword.fire.CarbCalc
CarbCalc           1         2    0.0425         9        11
End

!Exten:base Name:keyword.base.SpGroup
SPGROUP   spestab
FR JU PI PU SA SR LL TM PP PD WP LP VP HM RA
SPGROUP   spghw
FM BE RM SV SM BU BB SB AH HI CA HB RD DW PS &
AB AS WA BA GA HL LB HA HY BN WN SU YP MG CT &
MS MV ML AP MB WT BG TS HH SD RA SY CW BT BC &
BK WI SS BW EL WE AE RL OH OT
SPGROUP   spgoak
WO SO SK CB TO LK OV BJ SN CK WK CO RO QS PO &
BO LO
SPGROUP   spgsw
FR JU PI PU SP SA SR LL TM PP PD WP LP VP BY &
PC HM OS
!Exten:base Name:Compute_SpMcDBH
Compute            0
Mspestab = SPMCDBH(3,spestab,0,0,900,0,500,0)
Sspestab = SPMCDBH(9,spestab,0,0,900,0,500,0)
Mspghw = SPMCDBH(3,spghw,0,0,900,0,500,0)
Sspghw = SPMCDBH(9,spghw,0,0,900,0,500,0)
Mspgoak = SPMCDBH(3,spgoak,0,0,900,0,500,0)
Sspgoak = SPMCDBH(9,spgoak,0,0,900,0,500,0)
Mspgsw = SPMCDBH(3,spgsw,0,0,900,0,500,0)
Sspgsw = SPMCDBH(9,spgsw,0,0,900,0,500,0)
MALL = SPMCDBH(3,0,0,0,900,0,500,0)
SALL = SPMCDBH(9,0,0,0,900,0,500,0)
BA5b = SPMCDBH(2,0,0,0,5,0,500,0)
BA5a = SPMCDBH(2,0,0,5,999,0,999,0)
BTQMD = SQRT((BBA/BTPA)/0.005454)
RDBTH = BBA/SQRT(BTQMD)
CCPO = SPMCDBH(7,0,0,5,999,0,500,0)
End
!Exten:base Name:Compute_PreDefined
Compute            0
BTMERCHC = BTCuFt
ATMERCHC = ATCuFt
BTHBA = BBA
ATHBA = ABA
ATQMD = AADBH
End
DATABASE
!Exten:dbs Name:keyword.dbs.Summary
Summary            1
End
SPLabel
 All_FIA_Plots
Process

Stop

The output from these runs are cut and pasted below.

Year First day BBA Second day BBA Third day BBA 2014 128.9780883789 128.9780883789 128.9781188965 2019 138.2615966797 138.2615966797 138.2279510498 2022 145.4404144287 145.4404144287 145.4034881592 2027 154.1323089600 154.1323089600 154.4521179199 2032 162.9162597656 162.9162597656 163.4200286865 2037 167.2907867432 167.2907867432 167.2838592529 2042 166.9520721436 166.9520721436 166.9180145264 2047 166.6284179688 166.6284179688 166.5588226318 2052 166.5127105713 166.5127105713 166.4263153076 2057 166.4365844727 166.4365844727 166.2651214600 2062 166.3139190674 166.3139190674 166.0677642822 2067 166.1570281982 166.1570281982 165.8678741455 2072 166.0433502197 166.0433502197 165.7384338379 2077 165.8428955078 165.8428955078 165.5875091553 2082 165.6880950928 165.6880950928 165.4303741455 2087 165.5500946045 165.5500946045 165.2308197021 2092 165.4570007324 165.4570007324 165.0933990479 2097 165.3937683105 165.3937683105 164.9957427979 2102 165.3168029785 165.3168029785 164.8372650146 2107 165.3402557373 165.3402557373 164.8368530273 2112 165.2545166016 165.2545166016 164.7681732178 2117 165.2606201172 165.2606201172 164.7037048340 2122 165.2538452148 165.2538452148 164.6922149658

Despite no management, FVS generates different results for the same KEYWORDs when run on different days (as shown by the before-thinning basal area (BBA) from 2014 to 2122 in the FVS_Compute table. If run one day later, the results are exactly the same. If run 2 or more days later, the results are different.

Having different outputs with the same inputs leads to problems when we want to confirm that a given management is behaving as expected. Any help to bring more determinism to model results would be much appreciated!

DavidLRfs commented 9 months ago

Hello Wu,

I expect that I know exactly what is going on here. The database file for each of the runs is not the exact same file and the tree records are entering into FVS in a different order. Have the tree records in a different order does impact the simulation because the pseudo random numbers that get used do not get paired with the same tree records. I have seen this before when FVS simulations were being dynamically set up on a system and input databases for the run were being dynamically generated as well. What the user had to do is add a "oder by" clause to the SQL that reads the tree records were read into FVS ordered by the Tree ID. That kept the order of the tree records consistent between executions.

So, If you have output treelist files for these runs, compare them. You will probably find the individual tree record order is inconsitant.

Hope this helps. Lance

From: Wu Ma @.> Sent: Monday, January 22, 2024 2:17 PM To: USDAForestService/ForestVegetationSimulator @.> Cc: Subscribed @.***> Subject: [USDAForestService/ForestVegetationSimulator] Inconsistent FVS outputs when running on the same OS with the same keywords on different days (Issue #37)

I grew one FIA plot (PLT_CN = 218220253020004) from SC using the keywords pasted below for FVS version 20230929 in both Windows 11 and Linux (Ubuntu v22.04.3 LTS).

StdIdent

04520141103305700068 SCJG__MX

StandCN

218220253020004

MgmtId

A001

InvYear 2014

TimeInt 5

TimeInt 2 4

NumCycle 23

CYCLEAT 2023

CYCLEAT 2128

DATABASE

Summary 2

COMPUTDB 0 1

MisRpts 2

End

Treelist 0 0

Cutlist 0 0

DATABASE

TREELIDB 2

CUTLIDB 2

End

FMIn

CarbRept 2

CarbCut

CarbCalc 1 2 0.0425 9 11

FuelOut 0

FuelRept

End

DATABASE

CARBREDB 2

FUELREDB 2

FuelsOut 2

End

FMIn

SnagSum

DWDVlout

DWDCvOut

End

DATABASE

SNAGSUDB 2

DWDVLDB 2

DWDCVDB 2

End

StrClass

DATABASE

STRCLSDB 2

End

DelOTab 1

DelOTab 2

DelOTab 4

!Exten:base Name:From: FVS_GroupAddFilesAndKeywords

DATABASE

StandSQL

SELECT *

FROM FVS_StandInit_Plot

WHERE stand_cn = '%stand_cn%'

EndSQL

TreeSQL

SELECT *

FROM FVS_TreeInit_Plot

WHERE stand_cn ='%stand_cn%'

EndSQL

END

!Exten:base Name:keyword.base.SDICalc

SDICalc 0 0 1

FMIn

!Exten:fire Name:keyword.fire.CarbCalc

CarbCalc 1 2 0.0425 9 11

End

!Exten:base Name:keyword.base.SpGroup

SPGROUP spestab

FR JU PI PU SA SR LL TM PP PD WP LP VP HM RA

SPGROUP spghw

FM BE RM SV SM BU BB SB AH HI CA HB RD DW PS &

AB AS WA BA GA HL LB HA HY BN WN SU YP MG CT &

MS MV ML AP MB WT BG TS HH SD RA SY CW BT BC &

BK WI SS BW EL WE AE RL OH OT

SPGROUP spgoak

WO SO SK CB TO LK OV BJ SN CK WK CO RO QS PO &

BO LO

SPGROUP spgsw

FR JU PI PU SP SA SR LL TM PP PD WP LP VP BY &

PC HM OS

!Exten:base Name:Compute_SpMcDBH

Compute 0

Mspestab = SPMCDBH(3,spestab,0,0,900,0,500,0)

Sspestab = SPMCDBH(9,spestab,0,0,900,0,500,0)

Mspghw = SPMCDBH(3,spghw,0,0,900,0,500,0)

Sspghw = SPMCDBH(9,spghw,0,0,900,0,500,0)

Mspgoak = SPMCDBH(3,spgoak,0,0,900,0,500,0)

Sspgoak = SPMCDBH(9,spgoak,0,0,900,0,500,0)

Mspgsw = SPMCDBH(3,spgsw,0,0,900,0,500,0)

Sspgsw = SPMCDBH(9,spgsw,0,0,900,0,500,0)

MALL = SPMCDBH(3,0,0,0,900,0,500,0)

SALL = SPMCDBH(9,0,0,0,900,0,500,0)

BA5b = SPMCDBH(2,0,0,0,5,0,500,0)

BA5a = SPMCDBH(2,0,0,5,999,0,999,0)

BTQMD = SQRT((BBA/BTPA)/0.005454)

RDBTH = BBA/SQRT(BTQMD)

CCPO = SPMCDBH(7,0,0,5,999,0,500,0)

End

!Exten:base Name:Compute_PreDefined

Compute 0

BTMERCHC = BTCuFt

ATMERCHC = ATCuFt

BTHBA = BBA

ATHBA = ABA

ATQMD = AADBH

End

DATABASE

!Exten:dbs Name:keyword.dbs.Summary

Summary 1

End

SPLabel

All_FIA_Plots

Process

Stop

The output from these runs are cut and pasted below.

Year First day BBA Second day BBA Third day BBA 2014 128.9780883789 128.9780883789 128.9781188965 2019 138.2615966797 138.2615966797 138.2279510498 2022 145.4404144287 145.4404144287 145.4034881592 2027 154.1323089600 154.1323089600 154.4521179199 2032 162.9162597656 162.9162597656 163.4200286865 2037 167.2907867432 167.2907867432 167.2838592529 2042 166.9520721436 166.9520721436 166.9180145264 2047 166.6284179688 166.6284179688 166.5588226318 2052 166.5127105713 166.5127105713 166.4263153076 2057 166.4365844727 166.4365844727 166.2651214600 2062 166.3139190674 166.3139190674 166.0677642822 2067 166.1570281982 166.1570281982 165.8678741455 2072 166.0433502197 166.0433502197 165.7384338379 2077 165.8428955078 165.8428955078 165.5875091553 2082 165.6880950928 165.6880950928 165.4303741455 2087 165.5500946045 165.5500946045 165.2308197021 2092 165.4570007324 165.4570007324 165.0933990479 2097 165.3937683105 165.3937683105 164.9957427979 2102 165.3168029785 165.3168029785 164.8372650146 2107 165.3402557373 165.3402557373 164.8368530273 2112 165.2545166016 165.2545166016 164.7681732178 2117 165.2606201172 165.2606201172 164.7037048340 2122 165.2538452148 165.2538452148 164.6922149658

Despite no management, FVS generates different results for the same KEYWORDs when run on different days (as shown by the before-thinning basal area (BBA) from 2014 to 2122 in the FVS_Compute table. If run one day later, the results are exactly the same. If run 2 or more days later, the results are different.

Having different outputs with the same inputs leads to problems when we want to confirm that a given management is behaving as expected. Any help to bring more determinism to model results would be much appreciated!

- Reply to this email directly, view it on GitHubhttps://github.com/USDAForestService/ForestVegetationSimulator/issues/37, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AZVWHWZ2F5ATGKWUZESB7VDYP3JN3AVCNFSM6AAAAABCF4FOYOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA4TINZWG42DOMI. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

wuma2022 commented 9 months ago

Thanks, Lance, for your valuable information. I will implement it for several days and then check it to see if it is working.

wuma2022 commented 8 months ago

Hello Lance, Thanks again for your prompt responses. I ordered tree id and did get the exact same FVS outputs with same keywords. However, the outputs were changed slightly due to Operational System reset. Do you have any ideas how to address it? Thank you! Thanks, Wu

DavidLRfs commented 8 months ago

Hello Wu, I have no recommendations or suggestions and can only be confident that results from same exe or dll on the same system will be the same over and over again. We only have the Windows based system issued by the USDA Forest Service and that is how we develop, build and test the system we release. If a copy of the FVS code is compiled and linked on another platform and all options are as close as can be to what we use, I would expect the results to be the same to a certain degree. There is the definite possibility of some minor differences but are the differences significant? I suppose that depends on the point of view. So, whatever minor variation is going to exist will simply compound over every growth period. Have a good day. Lance