Amakihi from vignette, not converging in MRDS

[erex]

After converting the amakihi Distance project, I attempt to fit one of the models containing a covariate: hazard rate with hours after sunrise

> birds2 <- run_analysis(amakihi.proj$`p16 - HRpol HAS w82.5`)
Error in detfct.fit.opt(ddfobj, optim.options, bounds, misc.options, fitting = fitting) : 
  No convergence.

same result for any of the models that contain covariates.

[dill]

Looks like this was an mrds bug, can you install the gh version of mrds and give this another go?

On 04/06/2018 13:17, erex wrote:

After converting the amakihi Distance project, I attempt to fit one of the models containing a covariate: hazard rate with hours after sunrise

| > birds2 <- run_analysis(amakihi.proj$p16 - HRpol HAS w82.5) Error in detfct.fit.opt(ddfobj, optim.options, bounds, misc.options, fitting = fitting) : No convergence. |

same result for any of the models that contain covariates.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/DistanceDevelopment/readdst/issues/38, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAHofzfDyPUVipqiB_tSxxxqDHQFGPbks5t5SVbgaJpZM4UY-r4.

[erex]

I thought I have the gh version, but I'll do it again in the morning.

[erex]

Just re-installed mrds this morning, rerun amakihi analysis plucked from Distance project

> ama.hn.mas <- run_analysis(amakihi.proj$`m13 - HN MAS w82.5`, debug = TRUE)
Model name: m13 - HN MAS w82.5 
Call:
 mrds::ddf(dsmodel=~mcds(key="hn", formula=~MAS, adj.series="cos", adj.order=NULL), meta.data=list(width=82.5,left=0), control=list(), method="ds", data=data) 

Starting AIC adjustment term selection
Half-normal key 
  AIC = -8792.786
Half-normal key with order 2 cosine adjustments 
  AIC = 1287.314

Selected model:
  Half-normal key 
  AIC = -8792.786
Model name: m13 - HN MAS w82.5 
Call:
 mrds::ddf(dsmodel=~mcds(key="hn", formula=~MAS, adj.series="cos", adj.order=NULL), meta.data=list(width=82.5,left=0), control=list(), method="ds", data=data) 

Starting AIC adjustment term selection
Error in detfct.fit.opt(ddfobj, optim.options, bounds, misc.options) : 
  No convergence.
In addition: There were 50 or more warnings (use warnings() to see the first 50)

This is actually an odd model that contains (apparently) both a covariate and adjustment terms. Note the bizarre change in AIC score when adjustment terms are added.

Manually run ddf() on these data:

> ama.hn.mas.noadj <- ddf(dsmodel=~cds(key="hn", formula=~MAS),
+                         meta.data=list(width=82.5,left=0), 
+                         control=list(mono=TRUE, mono.strict=TRUE), method="ds", 
+                         data=amakihi.proj$`m13 - HN MAS w82.5`$env$data)
Error in detfct.fit.opt(ddfobj, optim.options, bounds, misc.options) : 
  No convergence.
In addition: There were 50 or more warnings (use warnings() to see the first 50)

[erex]

Snipping the same analysis from the Buckland et al (2015) case study website, where a standardisation is done on the covariate, no problem arises.

> min.hn <- fit.and.assess(key="hn", cov=c("MAS"), plot=TRUE)
Model contains covariate term(s): no adjustment terms will be included.
Fitting half-normal key function
AIC= 10826.537

[erex]

Following on from above, standardising covariate plucked from Distance project, then running through ddf() produces desired result,

ama.data.std <- amakihi.proj$`m13 - HN MAS w82.5`$env$data
ama.data.std$MAS.std <- ama.data.std$MAS/sd(ama.data.std$MAS, na.rm=TRUE)

ama.hn.mas.noadj.std <- ddf(dsmodel=~cds(key="hn", formula=~MAS.std),
                        meta.data=list(width=82.5,left=0), 
                        control=list(mono=TRUE, mono.strict=TRUE), method="ds", 
                        data=ama.data.std)

However, plotting is problematic

There is no argument to ddf() to stipulate point transect data are being fitted, so I'm not sure all is well. which looks like a detection function rather than a pdf, so request a pdf:

> plot(ama.hn.mas.noadj, pdf=TRUE)
Warning message:
In plot.ds(ama.hn.mas.noadj, pdf = TRUE) :
  Ignoring pdf=TRUE for line transect data

[erex]

This is nothing to do with readdst() so I should quit digging, but now I'm stuck on not understanding GOF and plotting of this point transect dataset:

nice.model.ds <- ds(key="hn", formula=~OBs+MAS.std, truncation=82.5, transect = "point"
                    data=ama.data.std)
tmp <- gof_ds(nice.model.ds, plot = TRUE, nboot=250, ks=TRUE)
plot(nice.model.ds, pdf=TRUE)

The plot from the final command is this (when I leave off the pdf=TRUE argument:

When I use the pdf=TRUE argument to plot() in which case I get:

> plot(nice.model.ds, pdf=TRUE)
Warning message:
In plot.ds(x$ddf, pl.den = pl.den, ...) :
  Ignoring pdf=TRUE for line transect data

I also note that I first used ds() without the transect="point" argument, then corrected myself. When I subject the two resulting objects to gof_ds(), the same values of the KS test statistic and the CvM test statistic were produced whether transect="point" was used. That seems unusual, but maybe I don't understand goodness of fit testing for point transects.

[dill]

Regarding points, not sure why this has gone wrong, as I get:

> amakihi.proj$`m13 - HN MAS w82.5`
Model name  : m13 - HN MAS w82.5
ID          : 179
Data filter : Sp=="COAM"
mrds call   : mrds::ddf(dsmodel=~mcds(key="hn", formula=~MAS, adj.series="cos", adj.order=NULL), meta.data=list(width=82.5,left=0,point=TRUE), control=list(), method="ds", data=data)

Very unclear where these issues are coming from. I'll note that there is a missing comma after "point" in your penultimate code block as a futile attempt to think this could be an easily solved problem...

[erex]

OK; don't know why RODBC doesn't find the fact of a point transect, but that would explain why subsequent calls to ddf() are operating under the illusion of line transect data.

[erex]

Adding the offending missing comma does not improve the situation

> ama.data.std <- amakihi.proj$`m13 - HN MAS w82.5`$env$data
> ama.data.std$MAS.std <- ama.data.std$MAS/sd(ama.data.std$MAS, na.rm=TRUE)
> ama.data.std$OBs <- as.factor(ama.data.std$OBs)
> 
> nice.model.ds <- ds(key="hn", formula=~OBs+MAS.std, truncation=82.5, transect = "point",
+                     data=ama.data.std)
Model contains covariate term(s): no adjustment terms will be included.
Fitting half-normal key function

All models failed to fit!

Error in ds(key = "hn", formula = ~OBs + MAS.std, truncation = 82.5, transect = "point",  : 
  No models could be fitted.

[dill]

Okay, I have my hands on a Windows ☠️ machine now and will test this out tomorrow.

[dill]

I think this is fixed in e308e81, could you give it a whirl?

[erex]

That seems to behave

> library(readdst)
> home.dir <- path.expand("~")
> sample.proj.dir <- paste0(home.dir, "/My Distance Projects/Sample Projects/")
> amakihi.proj.file <- paste0(sample.proj.dir, "Amakihi")
> amakihi.proj <- convert_project(amakihi.proj.file)
Loading required package: RODBC
> ama.data.std <- amakihi.proj$`m13 - HN MAS w82.5`$env$data
> ama.data.std$MAS.std <- ama.data.std$MAS/sd(ama.data.std$MAS, na.rm=TRUE)
> ama.data.std$OBs <- as.factor(ama.data.std$OBs)
> nice.model.ds <- ds(key="hn", formula=~OBs+MAS.std, truncation=82.5, transect = "point",
+                     data=ama.data.std)
Model contains covariate term(s): no adjustment terms will be included.
Fitting half-normal key function
AIC= 10803.875
No survey area information supplied, only estimating detection function.

> tmp <- gof_ds(nice.model.ds, plot = TRUE, nboot=250, ks=TRUE)

I note whenever I install_github("distancedevelopment/readdst") that mrds is also downloaded and recompiled. Is this necessary behaviour?

[dill]

install_github("distancedevelopment/readdst", upgrade_dependencies=FALSE) should solve this issue.

Closing this issue.