DistanceDevelopment / distance-bugs

A place to keep bugs in Distance
http://distancesampling.org/Distance
1 stars 0 forks source link

mismatch module error #158

Open Sid1245 opened 8 years ago

Sid1245 commented 8 years ago

The bug is within the output window below for MCDS analysis. I do not understand where the source of this error is. Please help. Thanks.

Initializing Making Data File Making Data Selection Queries Making Input File Engine started with the following command: C:\PROGRA~2\DISTAN~1\ec "C:\PROGRA~2\DISTAN~1\MCDS.exe 0, C:\Users\sid1245blake\AppData\Local\Temp\1\dst7558.tmp \options 2>C:\Users\sid1245\AppData\Local\Temp\1\dst7557.tmp" \ Error: Analysis engine terminated unexpectedly (with error code -1073741663). -- Start of Analyis Engine Log File -- This is mcds.exe version 6.2.0
Options;
Type=Point;
Distance=Radial /Measure='Meter';
Area /Units='Hectare';
Object=Cluster;
SF=1;
Selection=Sequential;
Lookahead=1;
Maxterms=5;
Bootstraps=999;
Seed=0;
Confidence=95;
Print=All;
End;
Data /Structure=Flat;
Fields=STR_LABEL, STR_AREA, SMP_LABEL, SMP_EFFORT, DISTANCE, SIZE, Estimation Type, Cloud Cover;
Factor /Name=Estimation Type /Levels=2 /Labels=D, I;
Factor /Name=Cloud Cover /Levels=4 /Labels=B, C, O, S;
Infile=C:\Users\sid1245\AppData\Local\Temp\1\dst7508.tmp /NoEcho;
Data will be input from file - [...]PDATA\LOCAL\TEMP\1\DST7508.TMP End;
Dataset has been stored. Estimate;
Distance /Intervals=0,30.25,60.5,90.75,233,364,486,607,729,850,971 /Width=971 /Left=0;
Density=All;
Density=Stratum /Design=Strata /Weight=Area;
Encounter=Stratum;
Detection=All;
Size=All;
Estimator /Key=HN /Adjust=CO /Criterion=AIC /Covariates=Estimation Type, Cloud Cover;
Estimator /Key=HN /Adjust=PO /Criterion=AIC /Covariates=Estimation Type, Cloud Cover;
Estimator /Key=HN /Adjust=HE /Criterion=AIC /Covariates=Estimation Type, Cloud Cover;
Estimator /Key=HA /Adjust=CO /Criterion=AIC /Covariates=Estimation Type, Cloud Cover;
Estimator /Key=HA /Adjust=PO /Criterion=AIC /Covariates=Estimation Type, Cloud Cover;
Monotone=None;
Pick=AIC;
GOF;
Cluster /Bias=GXLOG;
Bootstrap /Instratum /Samples;
VarN=Empirical;
End;
\ Warning: Estimation routine failed to converge as some estimates of the pdf are negative in iteration 14. Using results from previous iteration.
\ Warning: convergence failure \ Warning: Exact distance values, rather than distance intervals, have been used in size bias regression calculations. \ Internal Error 53: Mismatched modules - 2 0 0 0 \ Internal Error 54: Mismatched stats - 4 0 0 0 2

-- End of Analyis Engine Log File -- \ Warning: Analysis engine wrote some output to the console error stream, reproduced below. ** -- Start of Console Error Stream -- forrtl: severe (161): Program Exception - array bounds exceeded

Image PC Routine Line Source
MCDS.exe 00438ED4 GETPARMS 200 Dmod.for

-- End of Console Error Stream -- Analysis engine status 'Run failed - analysis engine encountered an unknown error, possibly in the input or output files' Reading Results Run finished

erex commented 8 years ago

Sid1245

Thanks for providing the program log in your report. The way I diagnose this problem is that you are fitting a detection function with 2 factor covariates: one with 2 levels, the other with 4 levels. You are fitting 5 detection functions to this amalgamation, then you are bootstrapping the lot.

The first error message generated (the rest are probably derivatives of the first error) is "convergence failure". That is the nature of the problem more likely a feature of the data (and fitting requested) than a program bug.

Point transect detection functions are notorious to fit (lacking detections in the region where fitting is most sensitive). Atop this challenge you are adding covariate (another challenge--convergence problems), atop this your covariates are factor rather than continuous (a good chance that of the 8 covariate combinations several will have almost no observations, making the fitting problem harder still), then you add adjustment terms onto the covariate models (a bad idea, I'd remove the adjustment terms), then you ask for this to happen to 5 different model types each with 999 bootstrap replicates.

It is hard to imagine a data set for which a convergence failure would not result from this request.

Summary: it is not a program bug, rather fitting of a set of models that are too complex (as a result, this conversation should have happened on the distance Google-groups email list rather than in the Github bug report). Remedy for you, a) remove adjustment terms with covariates in the detection function, b) check your data for 'structural zeros' how many data are supporting each of your 8 covariate factor-level combinations (probably few in some instances, suggesting you merge some factor levels to give some data support to the models you are trying to fit).

Covariate curve-fitting is difficult, you have shown how difficult it is. Give Distance some help by determining how plausible it is to fit what you ask, and how much data you have provided for the task.

After employing the suggested remedies, take this entire conversation thread to the Google-group so others can benefit from your experiences.

Sid1245 commented 8 years ago

Thanks for your quick response. I apologize for posting in the wrong place. I had 3 years of data and didn't think that I had limited data that would create the error I observed.I had more than 30 observations per combination of levels. But, I believe my confusion was that the error only happened when dividing by stratum (3 sites and multiple years). When choosing no stratification, it worked without error. It might have been the stratum division combined with what you proposed above in addition with the top model choices (# of bins, truncation) that created the error and my confusion. I wish I understood the minimum numbers needed exactly so I operate better proactively versus reactively.

While I do not understand the fine detail, I do understand the jist of the error. Thanks for that.