ADAM + greybox::ro() + .GlobalEnv [SOLVED]

[Steviey]

R version 4.0.2, smooth version 3.0.1

Hello,

after updating to smooth 3.0.1, I can't get it running anymore, with greybox::ro(),... what is wrong?... Code::

idx=120
y<-rnorm(idx, mean=15, sd=5)
x<-cbind(
    x1 =  rnorm(idx, mean=15, sd=5)
    ,x2 =  rnorm(idx, mean=15, sd=5)
)    

fitXregs    <- data.frame(x=x,y=y,stringsAsFactors=F)
fitXregs    <- fitXregs %>% dplyr::relocate(y)

ourCall <- paste0("forecast(auto.adam(data=data
,model=",paste0("'",as.character('MAN'),"'"),"
,lags=",as.numeric(1),"
,ic=",paste0("'",as.character('AICc'),"'"),"
,regressors=",paste0("'",as.character('select'),"'"),"
,distribution=",paste0("'",as.character('dinvgauss'),"'"),"
,initial=",paste0("'",as.character(params$initial),"'"),"
,bounds=",paste0("'",as.character('usual'),"'"),"
,occurrence=",paste0("'",as.character('none'),"'"),"
,silent=",as.logical(FALSE),"
,orders=list(select=",as.logical(FALSE),")
), h=h, level=0.95, interval='prediction')")

ourVal      <-  c("mean")

error<-tryCatch({
    myFit<-  greybox::ro(fitXregs, h=1, origins=3, call=ourCall, value=ourVal, ci=TRUE)
    error<-0
}, warning = function(w) {
    print(w)
    return(1)
}, error = function(e) {
    print(e)
    return(1)
})

Error: <simpleError in attr(x, "tsp") <- c(1, NROW(x), 1): invalid time series parameters specified>

[config-i1]

This is a very weird bug, somehow related to forecast function. This code doesn't work as well (it's easier, and a bit more appropriate, e.g. ro() assumes that we work with a vector in data):

y<-rnorm(idx, mean=15, sd=5) x<-cbind( x1 = rnorm(idx, mean=15, sd=5) ,x2 = rnorm(idx, mean=15, sd=5) ) fitXregs <- data.frame(y=y,x=x) ourCall <- "forecast(adam(fitXregs[counti,],model='MAN',lags=1,regressors='select',distribution='dinvgauss'), newdata=fitXregs[counto,], h=h, level=0.95)" ourVal <- "mean" myFit<- ro(fitXregs$y, h=1, origins=3, call=ourCall, value=ourVal, ci=TRUE)

But changing the call to this (specific method) fixes it:

ourCall <- "forecast.adam(adam(fitXregs[counti,],model='MAN',lags=1,regressors='select',distribution='dinvgauss'), newdata=fitXregs[counto,], h=h, level=0.95)"

I don't know why this happens. Could be a weird treatment of S3 class, but I don't see a reason for it.

Also, just to check, have you tried the latest github version of smooth, v3.0.2.41009?

[config-i1]

Okay. It seems to work on the latest version on github. Can you, please, check?

[Steviey]

I'm afraid this could take some time...

Error: Failed to install 'smooth' from GitHub:
(converted from warning) incomplete final line found on '/home/rstud/.Rprofile' 

... Update: Rprofile needs an empty line as 'ending' (funny R-thingy) ... working on it...

[Steviey]

R version 4.0.2, smooth version 3.0.2.41009 (dev)

Hm, can you point out the notation of "counti/counto" a little bit?... I made the mistake to not to change the method...still trying... comparing this https://www.rdocumentation.org/packages/greybox/versions/0.6.4/topics/ro and this... https://cran.r-project.org/web/packages/greybox/vignettes/ro.html I would guess "counti/counto" is a placeholder for an internal function, which is little hard to understand in the first documentation. Actually my target is "auto.adam()", but even with adam() it seems not to work right, so far...(could not find function "forecast.adam")

idx=120
  y<-rnorm(idx, mean=15, sd=5)
  x<-cbind(
      x1 =  rnorm(idx, mean=15, sd=5)
      ,x2 =  rnorm(idx, mean=15, sd=5)
  )    

  fitXregs    <- data.frame(x=x,y=y,stringsAsFactors=F)
  fitXregs    <- fitXregs %>% dplyr::relocate(y)

  ourCall <- paste0("forecast.adam(adam(fitXregs[counti,]
  ,model=",paste0("'",as.character('MAN'),"'"),"
  ,lags=",as.numeric(1),"
  ,ic=",paste0("'",as.character('AICc'),"'"),"
  ,regressors=",paste0("'",as.character('select'),"'"),"
  ,distribution=",paste0("'",as.character('dinvgauss'),"'"),"
  ,initial=",paste0("'",as.character(params$initial),"'"),"
  ,bounds=",paste0("'",as.character('usual'),"'"),"
  ,occurrence=",paste0("'",as.character('none'),"'"),"
  ,silent=",as.logical(FALSE),"
  ,orders=list(select=",as.logical(FALSE),")
  ),newdata=fitXregs[counto,],h=h, level=0.95)")

  ourVal      <-  c("mean")

  error<-tryCatch({
      myFit<-  greybox::ro(fitXregs$y, h=1, origins=3, call=ourCall, value=ourVal, ci=TRUE)
      error<-0
  }, warning = function(w) {
      print(w)
      return(1)
  }, error = function(e) {
      print(e)
      return(1)
  })

Error: <simpleError in forecast.adam(adam(fitXregs[counti, ], model = "MAN", lags = 1, ic = "AICc", regressors = "select", distribution = "dinvgauss", initial = "optimal", bounds = "usual", occurrence = "none", silent = FALSE, orders = list(select = FALSE)), newdata = fitXregs[counto, ], h = h, level = 0.95): could not find function "forecast.adam">

By the way, the 'non-ro()-version' (simple if-switch in my code) works well on my system.

[config-i1]

I've checked this on a couple of PCs, and the code above works with "forecast" instead of "forecast.adam" on smooth v3.0.2.41009 and greybox v0.6.7.41002, when I specify initial="optimal" (I don't have params, and it should only be either "optimal" or "backcasting" in auto.adam).

As for counti and counto, they are not explained explicitly, but they are described both in examples of ro() and in the vignette. I should probably add them in the description of the parameters of the function. Point taken.

[Steviey]

Ubuntu 16.x, R version 4.0.2, smooth version 3.0.2.41009 (dev), greybox 0.6.7.41002 Strange... I updated greybox and all its dependencies too. The same code with mehtod: 'forecast' gives me: <simpleError in is.adam.sim(data): object 'fitXregs' not found>

idx=120
y<-rnorm(idx, mean=15, sd=5)
x<-cbind(
    x1 =  rnorm(idx, mean=15, sd=5)
    ,x2 =  rnorm(idx, mean=15, sd=5)
)    

fitXregs    <- data.frame(x=x,y=y,stringsAsFactors=F)
fitXregs    <- fitXregs %>% dplyr::relocate(y)

ourCall <- paste0("forecast(adam(fitXregs[counti,]
,model=",paste0("'",as.character('MAN'),"'"),"
,lags=",as.numeric(1),"
,ic=",paste0("'",as.character('AICc'),"'"),"
,regressors=",paste0("'",as.character('select'),"'"),"
,distribution=",paste0("'",as.character('dinvgauss'),"'"),"
,initial=",paste0("'",as.character('optimal'),"'"),"
,bounds=",paste0("'",as.character('usual'),"'"),"
,occurrence=",paste0("'",as.character('none'),"'"),"
,silent=",as.logical(FALSE),"
,orders=list(select=",as.logical(FALSE),")
),newdata=fitXregs[counto,],h=h, level=0.95)")

ourVal      <-  c("mean")

error<-tryCatch({
    myFit<-  greybox::ro(fitXregs$y, h=1, origins=3, call=ourCall, value=ourVal, ci=TRUE)
    error<-0
}, warning = function(w) {
    print(w)
    return(1)
}, error = function(e) {
    print(e)
    return(1)
})

#View(fitXregs)
#View(params)

if(error==0){
    View(myFit)
    if(length(myFit$mean)>0 && error==0){
        myFit$mean      <-  as.vector(as.numeric(myFit$mean))
        myFit$holdout   <-  as.vector(as.numeric(myFit$holdout))
        testRmse        <-  MSE(myFit$holdout, myFit$mean)
        testRmse        <-  sqrt(testRmse)
        testRmse        <-  as.numeric(testRmse)
        results[ee]     <-  round(testRmse,2)
    }else{
        results[ee]     <- 1000
    }
}else{
    results[ee]     <-  1000
}

[config-i1]

Just a suggestion, why not simplify the code first and see if it works (this should be equivalent to yours):

idx=120
y<-rnorm(idx, mean=15, sd=5)
x<-cbind(
    x1 =  rnorm(idx, mean=15, sd=5)
    ,x2 =  rnorm(idx, mean=15, sd=5)
)
fitXregs    <- data.frame(y=y,x=x,stringsAsFactors=F)

ourCall <- "forecast(adam(fitXregs[counti,],model='MAN',lags=1,ic='AICc',
regressors='select',distribution='dinvgauss',initial='optimal',bounds='usual',
occurrence='none',silent=FALSE,orders=list(select=FALSE)),
newdata=fitXregs[counto,],h=h, level=0.95)"

ourVal <- "mean"

myFit<-  greybox::ro(fitXregs$y, h=1, origins=3, call=ourCall, value=ourVal, ci=TRUE)

This works on my PC (Kubuntu 20.04). See if this works for you. If it does, maybe there are some typos in the original one.

Also, make sure that "y" doesn't contain negative values - Inverse Gaussian doesn't work with them.

In addition, I would recommend switching silent to TRUE for ro(), otherwise it will start plotting tones of figures.

Finally, you don't need to provide all the default parameters, so the call can be simplified to:

ourCall <- "forecast(adam(fitXregs[counti,], model='MAN', regressors='select', distribution='dinvgauss'), newdata=fitXregs[counto,], h=h, level=0.95)"

[Steviey]

Ubuntu 16.x, R version 4.0.2, smooth version 3.0.2.41009 (dev), greybox 0.6.7.41002 (dev)

My guess is, that it has to do with: 'In this case we just need to make sure that the Global Environment contains the xreg data frame.' Honestly, if I look in the Global Environment tab of RStudio, I see no fitXreg variable. ls() shows the same. Until now I thought it will be automatically registered there. But I have to mention, that my script weights more then 3 MB and there is a ton of stuff already registered in the Global Environment. It is also an external S4 class involved.

Update: This seems to do the trick:

.GlobalEnv$fitXregs <- fitXregs

I have no clue why the variable is not automatically registered in the Global Environment. There are some thoughts about it here, but no real answer for this case... https://stackoverflow.com/questions/25841549/maximum-number-of-objects-in-r-workspace

[config-i1]

This is really weird. When you create a variable via fitXregs <- data.frame(x=x,y=y,stringsAsFactors=F), it is automatically created in the Global Environment. Could this be something that dplyr::relocate(y) does?

[Steviey]

I have no evidence of such a behavior of dplyr (double checked). By the way, during checking, the var was globally registered- before explicitly setting it global (fresh R-session). Therefore it also could be a temporal problem.