Nested hook execution in package "covmat"

kondziu commented 6 years ago

I got the following log:

[1/1] Executing file: /media/kondziu/b7a9548c-13a0-4fe6-8a28-de8d491a209b/R/traces/lucoa/2018-01-19-18-30-19/vignettes/covmat-CovarianceEstimation.R
Loading required package: mvtnorm
Loading required package: RMTstat
Loading required package: grid
Loading required package: zoo

Attaching package: âzooâ

The following objects are masked from âpackage:baseâ:

    as.Date, as.Date.numeric

Attaching package: âxtsâ

The following objects are masked from âpackage:dplyrâ:

    first, last

Loading required package: fit.models
Loading required package: foreach
Loading required package: PerformanceAnalytics

Attaching package: âPerformanceAnalyticsâ

The following object is masked from âpackage:graphicsâ:

    legend

Loading required package: rugarch
Loading required package: parallel
Error in È(4$}U : 
  [ERROR] - [NESTED HOOK EXECUTION] - probe_promise_created triggers probe_jump_ctxt
Calls: library ... cpp_refMethods -> sapply -> lapply -> FUN -> <Anonymous> -> paste
In addition: Warning message:
In  ±È)}U :
  '/media/kondziu/b7a9548c-13a0-4fe6-8a28-de8d491a209b/R/traces/lucoa/2018-01-19-18-30-19/vignettes' already exists
Error in È(4$}U : 
  [ERROR] - [NESTED HOOK EXECUTION] - probe_promise_created triggers probe_jump_ctxt
Calls: library ... asNamespace -> loadNamespace -> namespaceImport -> loadNamespace
Error in È(4$}U : 
  [ERROR] - [NESTED HOOK EXECUTION] - probe_promise_created triggers probe_jump_ctxt
Calls: library ... namespaceImportFrom -> asNamespace -> loadNamespace
Error in È(4$}U : 
  [ERROR] - [NESTED HOOK EXECUTION] - probe_promise_created triggers probe_jump_ctxt
Calls: library ... tryCatchList -> tryCatchOne -> doTryCatch -> loadNamespace
error while dyntracing code block
Error: [ERROR] - [NESTED HOOK EXECUTION] - probe_promise_created triggers probe_end
Execution halted
Error: [ERROR] - [NESTED HOOK EXECUTION] - probe_promise_created triggers probe_promise_created

The code of the vignette (including instrumentation):

library(promisedyntracer)

dyntracer <- create_dyntracer('/media/kondziu/b7a9548c-13a0-4fe6-8a28-de8d491a209b/R/traces/lucoa/2018-01-19-18-30-19/data/covmat-CovarianceEstimation.sqlite',verbose=FALSE)
dyntrace(dyntracer, {

    ## ----setup, include=FALSE------------------------------------------------

    knitr::opts_chunk$set(fig.pos="h")
    knitr::opts_chunk$set(cache.path='./CovarianceEstimation_cache/')

    ## ----load_packages, results='hide', echo=FALSE, warning=FALSE, error=FALSE, message=FALSE----
    library(knitcitations)
    cleanbib()
    options("citation_format" = "pandoc")

    library(covmat)
    library(xts)
    library(robust)
    library(PortfolioAnalytics)
    library(rmgarch)

    ## ----load, eval=FALSE----------------------------------------------------
    #  library(devtools)
    #  install_github("arorar/covmat")

    ## ----doi, echo=FALSE, warning=FALSE, error=FALSE, message=FALSE----------
    bib <- read.bibtex("references.bib")

    ## ----rmt-load------------------------------------------------------------

    data("dow30data")

    ## ---- eval=FALSE---------------------------------------------------------
    #  
    #  estRMT(R, Q =NA, cutoff = c("max", "each"),
    #         eigenTreat = c("average", "delete") ,
    #         numEig=1, parallel = TRUE)
    #  

    ## ----rmt-est, echo=FALSE, warning=FALSE, error=FALSE, message=FALSE------
    model <- estRMT(dow30data, parallel=FALSE)

    ## ----rmt-plot, fig.width=8, fig.height=4, fig.keep='last'----------------

    plot(model)

    ## ----rmt-custommoment----------------------------------------------------

    custom.portfolio.moments <- function(R, portfolio) {
      momentargs <- list()
      momentargs$mu  <-  matrix(as.vector(apply(R,2, "mean")), ncol = 1)
      momentargs$sigma  <-  estRMT(R, parallel=FALSE)$cov
      momentargs$m3 <- matrix(0, nrow=ncol(R), ncol=ncol(R)^2)
      momentargs$m4 <- matrix(0, nrow=ncol(R), ncol=ncol(R)^3)

      return(momentargs)
    }

    ## ----rmt-portfoliospec---------------------------------------------------

    pspec.lo <- portfolio.spec(assets = colnames(dow30data))

    #long-only
    pspec.lo <- add.constraint(pspec.lo, type="full_investment")
    pspec.lo <- add.constraint(pspec.lo, type="long_only")

    pspec.lo <- add.objective(portfolio=pspec.lo, type="return", name="mean")
    pspec.lo <- add.objective(portfolio=pspec.lo, type="risk", name="var")

    ## ----rmt-run, warning=FALSE, error=FALSE, message=FALSE, eval = FALSE----
    #  
    #  opt.ordinary <-
    #    optimize.portfolio.rebalancing(dow30data, pspec.lo,
    #                                   optimize_method="quadprog",
    #                                   rebalance_on="months",
    #                                   training_period=120,
    #                                   trailing_periods=120)
    #  opt.rmt <-
    #    optimize.portfolio.rebalancing(dow30data, pspec.lo,
    #                                   optimize_method="quadprog",
    #                                   momentFUN = "custom.portfolio.moments",
    #                                   rebalance_on="months",
    #                                   training_period=120,
    #                                   trailing_periods=120)

    ## ----rmt-results, eval = FALSE-------------------------------------------
    #  
    #  ordinary.wts <- na.omit(extractWeights(opt.ordinary))
    #  ordinary <- Return.rebalancing(R=dow30data, weights=ordinary.wts)
    #  
    #  rmt.wts <- na.omit(extractWeights(opt.rmt))
    #  rmt <- Return.rebalancing(R=dow30data, weights=rmt.wts)
    #  
    #  rmt.strat.rets <- merge.zoo(ordinary,rmt)
    #  colnames(rmt.strat.rets) <- c("ordinary", "rmt")
    #  

    ## ----rmt-results-main,  results='hide', echo=FALSE, warning=FALSE, error=FALSE, message=FALSE----

    filepath <- "./CovarianceEstimation_cache/rmt_strategy_rets.RData"

    if (file.exists(filepath)) {
      load(filepath)
    } else {
      opt.ordinary <- 
      optimize.portfolio.rebalancing(dow30data, pspec.lo, 
                                     optimize_method="quadprog",
                                     rebalance_on="months", 
                                     training_period=120,
                                     trailing_periods=120)
      opt.rmt <- 
        optimize.portfolio.rebalancing(dow30data, pspec.lo, 
                                       optimize_method="quadprog",
                                       momentFUN = "custom.portfolio.moments",
                                       rebalance_on="months", 
                                       training_period=120, 
                                       trailing_periods=120)

      ordinary.wts <- na.omit(extractWeights(opt.ordinary))
      ordinary <- Return.rebalancing(R=dow30data, weights=ordinary.wts)

      rmt.wts <- na.omit(extractWeights(opt.rmt))
      rmt <- Return.rebalancing(R=dow30data, weights=rmt.wts)

      rmt.strat.rets <- merge.zoo(ordinary,rmt)
      colnames(rmt.strat.rets) <- c("ordinary", "rmt")

      save(rmt.strat.rets, file = filepath)
    }

    ## ----rmtstratplots-------------------------------------------------------
    charts.PerformanceSummary(rmt.strat.rets,wealth.index = T, 
                              colorset = c("red","blue"), 
                              main=paste(c("Comparison of Portflio ",
                                         "Performance using two ",
                                         "different covariance matrices"),
                                         collapse=""), cex.legend = 1.3, 
                              cex.axis = 1.3, legend.loc = "topleft")

    ## ----isdcc-load----------------------------------------------------------

    data("etfdata")

    ## ---- eval=FALSE---------------------------------------------------------
    #  
    #  isdccfit(R, numRegimes, transMatbounds = c(2,10),
    #           dccBounds = c(0,1), w = NA, ...)
    #  

    ## ----isdcc-est, echo=FALSE, warning=FALSE, error=FALSE, message=FALSE, eval=FALSE----
    #  model.isdcc <- isdccfit(etfdata, numRegimes=3, parallelType = 1, itermax = 100)
    #  

    ## ----isdcc-est-main, results='hide' ,echo=FALSE, warning=FALSE, error=FALSE, message=FALSE----

    filepath <- "./CovarianceEstimation_cache/isdcc_model.RData"
    if (file.exists(filepath)) {
      load(filepath)
    } else {
      model.isdcc <- isdccfit(etfdata, numRegimes=3, parallelType = 1, itermax = 100)
      save(model.isdcc, file = filepath)
    }

    ## ----isdcc-plot, fig.width=8, fig.height=4, fig.keep='last'--------------

    plot(model.isdcc)

    ## ----isdcc-custommoment--------------------------------------------------

    custom.portfolio.moments.isdcc <- function(R, portfolio) {

      momentargs <- list()
      momentargs$mu  <-  matrix(as.vector(apply(R,2, "mean")), ncol = 1)

      result <- isdccfit(R, numRegimes = 2, itermax=100)
      ldate <- as.character(tail(index(R),1))
      maxProbIndex <- which.max(result$filtProb[ldate,])
      momentargs$sigma  <-  result$cov[[ldate]][[maxProbIndex]]

      momentargs$m3 <- matrix(0, nrow=ncol(R), ncol=ncol(R)^2)
      momentargs$m4 <- matrix(0, nrow=ncol(R), ncol=ncol(R)^3)

      return(momentargs)
    }

    custom.portfolio.moments.dcc <- function(R, portfolio) {

      momentargs <- list()
      momentargs$mu  <-  matrix(as.vector(apply(R,2, "mean")), ncol = 1)

      garch11.spec <- ugarchspec(mean.model = list(armaOrder = c(0,0)), 
                                variance.model = list(garchOrder = c(1,1)
                                              , model = "sGARCH"), 
                                distribution.model = "norm")

      dcc.garch11.spec <- dccspec(uspec = multispec( replicate(ncol(R), 
                                                      garch11.spec) ), 
                                 dccOrder = c(1,1), distribution = "mvnorm")

      dcc.fit <- dccfit(dcc.garch11.spec, data = R)
      momentargs$sigma  <-  rcov(dcc.fit)[,,as.character(tail(index(R),1))]

      momentargs$m3 <- matrix(0, nrow=ncol(R), ncol=ncol(R)^2)
      momentargs$m4 <- matrix(0, nrow=ncol(R), ncol=ncol(R)^3)

      return(momentargs)
    }

    ## ----isdcc-portfoliospec-------------------------------------------------

    datap <- etfdata["2009-07/"]

    pspec.lo.isdcc <- portfolio.spec(assets = colnames(datap))

    #long-only
    pspec.lo.isdcc <- add.constraint(pspec.lo.isdcc, type="full_investment")
    pspec.lo.isdcc <- add.constraint(pspec.lo.isdcc, type="long_only")

    pspec.lo.isdcc <- add.objective(portfolio=pspec.lo.isdcc, 
                                    type="return", name="mean")
    pspec.lo.isdcc <- add.objective(portfolio=pspec.lo.isdcc, 
                                    type="risk", name="var")

    ## ----isdcc-run, warning=FALSE, error=FALSE , message=FALSE, results='hide', eval=FALSE----
    #  
    #  ordinary <-
    #    optimize.portfolio.rebalancing(datap, pspec.lo.isdcc,
    #                                   optimize_method="quadprog",
    #                                   rebalance_on="months",
    #                                   training_period=120,
    #                                   trailing_periods=120)
    #  
    #  opt.dcc <-
    #    optimize.portfolio.rebalancing(datap, pspec.lo.isdcc,
    #                                    optimize_method="quadprog",
    #                                    momentFUN =
    #                                     "custom.portfolio.moments.dcc",
    #                                    rebalance_on="months",
    #                                    training_period=120,
    #                                    trailing_periods=120)
    #  
    #  opt.isdcc <-
    #    optimize.portfolio.rebalancing(datap, pspec.lo.isdcc,
    #                                            optimize_method="quadprog",
    #                                            momentFUN =
    #                                     "custom.portfolio.moments.isdcc",
    #                                            rebalance_on="months",
    #                                            training_period=120,
    #                                            trailing_periods=120)

    ## ----isdcc-results,  eval=FALSE------------------------------------------
    #  
    #  ord.wts <- na.omit(extractWeights(ordinary))
    #  ord <- Return.rebalancing(R=datap, weights=ord.wts)
    #  
    #  dcc.wts <- na.omit(extractWeights(opt.dcc))
    #  dcc <- Return.rebalancing(R=datap, weights=dcc.wts)
    #  
    #  isdcc.wts <- na.omit(extractWeights(opt.isdcc))
    #  isdcc <- Return.rebalancing(R=datap, weights=isdcc.wts)
    #  
    #  isdcc.strat.rets <- merge.zoo(merge.zoo(ord, dcc), isdcc)
    #  colnames(isdcc.strat.rets) <- c("ordinary", "dcc", "isdcc")
    #  

    ## ----isdcc-results-main,  results='hide', echo=FALSE, warning=FALSE, error=FALSE, message=FALSE----

    filepath <- "./CovarianceEstimation_cache/isdcc_strategy_rets.RData"
    if (file.exists(filepath)) {
      load(filepath)
    } else {
      ordinary <- 
      optimize.portfolio.rebalancing(datap, pspec.lo.isdcc, 
                                     optimize_method="quadprog",
                                     rebalance_on="months", 
                                     training_period=120,
                                     trailing_periods=120)

      opt.dcc <- 
        optimize.portfolio.rebalancing(datap, pspec.lo.isdcc, 
                                        optimize_method="quadprog",
                                        momentFUN = 
                                         "custom.portfolio.moments.dcc",
                                        rebalance_on="months",
                                        training_period=120,
                                        trailing_periods=120)

      opt.isdcc <- 
        optimize.portfolio.rebalancing(datap, pspec.lo.isdcc, 
                                                optimize_method="quadprog",
                                                momentFUN = 
                                         "custom.portfolio.moments.isdcc",
                                                rebalance_on="months",
                                                training_period=120,
                                                trailing_periods=120)

      ord.wts <- na.omit(extractWeights(ordinary))
      ord <- Return.rebalancing(R=datap, weights=ord.wts)

      dcc.wts <- na.omit(extractWeights(opt.dcc))
      dcc <- Return.rebalancing(R=datap, weights=dcc.wts)

      isdcc.wts <- na.omit(extractWeights(opt.isdcc))
      isdcc <- Return.rebalancing(R=datap, weights=isdcc.wts)

      isdcc.strat.rets <- merge.zoo(merge.zoo(ord, dcc), isdcc)
      colnames(isdcc.strat.rets) <- c("ordinary", "dcc", "isdcc")

      save(isdcc.strat.rets, file = filepath)
    }

    ## ----isdccstratplots, fig.width=10, fig.height=8-------------------------
    charts.PerformanceSummary(isdcc.strat.rets,wealth.index = T,
                              colorset = c("red","blue","green"), 
                              main=paste(c("Comparison of Portflio ",
                                         "Performance using two ",
                                         "different covariance matrices"),
                                         collapse=""), cex.legend = 1.3, 
                              cex.axis = 1.3, legend.loc = "topright") 

    ## ----rmtdata-load--------------------------------------------------------

    data("rmtdata")

    ## ---- eval=FALSE---------------------------------------------------------
    #  
    #  estSpikedCovariance(R, gamma = NA,
    #                        numOfSpikes = NA,
    #                        method = c("KNTest", "median-fitting"),
    #                        norm = c("Frobenius", "Operator", "Nuclear"),
    #                        pivot = 1, statistical = NA,
    #                        fit = NA)
    #  

    ## ----spike-plot, eval=FALSE----------------------------------------------
    #  
    #    plotSpikedCovariance(rmtdata)
    #  

    ## ----spike-plot-main,  results='hide',  echo=FALSE, warning=FALSE, error=FALSE, message=FALSE, fig.show='hide'----

    filepath <- "./CovarianceEstimation_cache/Shrinkage.png"
    if (!file.exists(filepath)) {
      png(file = filepath, width = 12, height = 10, units = "in", res = 300)
      plotSpikedCovariance(rmtdata)
      dev.off()
    }

    ## ---- eval=FALSE---------------------------------------------------------
    #  
    #  robustMultExpSmoothing(R, smoothMat = NA, startup_period = 10,
    #                           training_period = 60 , seed = 9999, trials = 50,
    #                           method = "L-BFGS-B", lambda = 0.2)
    #  

    ## ----croux-est, eval=FALSE-----------------------------------------------
    #  
    #  data("dow30data")
    #  symbols <- c('AAPL', 'GE', 'MSFT' , 'NKE', 'V')
    #  
    #  R <- dow30data[,which(colnames(dow30data) %in% symbols)]
    #  smoothfit <- robustMultExpSmoothing(R)
    #  

    ## ----croux-est-main, results='hide', echo=FALSE, warning=FALSE, error=FALSE, message=FALSE----
    library(optimx)

    data("dow30data")
    symbols <- c('AAPL', 'GE', 'MSFT' , 'NKE', 'V')

    R <- dow30data[,which(colnames(dow30data) %in% symbols)]

    filepath <- "./CovarianceEstimation_cache/croux_smoothfit.RData"
    if (file.exists(filepath)) {
      load(filepath)
    } else {
      smoothfit <- robustMultExpSmoothing(R) 
      save(smoothfit, file = filepath)
    }

    ## ----croux-plots1, fig.width=10, fig.height=6----------------------------

    plotmissing(R)

    ## ----croux-plots2, fig.width=10, fig.height=6----------------------------

    plotmissing(smoothfit$cleanValues)

    ## ----croux-cov-plots, fig.width=10, fig.height=6-------------------------

    rob <- covRob(R)$cov
    compareCov(smoothfit$covMat, rob, labels = c("Robust Croux", "Robust"))

    ## ----references, echo=FALSE, warning=FALSE, error=FALSE, message=FALSE, echo=FALSE----
    #write.bibtex(file="references.bib")

})
destroy_dyntracer(dyntracer)
write('OK', '/media/kondziu/b7a9548c-13a0-4fe6-8a28-de8d491a209b/R/traces/lucoa/2018-01-19-18-30-19/data/covmat-CovarianceEstimation.OK')

aviralg commented 6 years ago

Here is the gdb backtrace for this vignette -

#0  0x0000000000476d77 in Rf_findcontext (mask=4, env=0x299faba0, val=0x2ee6c878) at context.c:331
#1  0x00000000004d193b in gotoExitingHandler (cond=0xa499d8, call=0x3989910, entry=0x2ee4f748) at errors.c:1621
#2  0x00000000004d1b99 in vsignalError (call=0x3989910, format=0x648055 "%s", ap=0x7ffffff99d58) at errors.c:1662
#3  0x00000000004cefeb in Rf_errorcall (call=0x3989910, format=0x648055 "%s") at errors.c:794
#4  0x00000000004cf328 in Rf_error (format=0x7fffe0c0d240 "DYNTRACE: encountered ACTIVE BINDING frame, missing implementation, confused") at errors.c:846
#5  0x00007fffe0b434a3 in get_binding_value (frame=<optimized out>) at lookup.cpp:15
#6  get_hash (table=<optimized out>, symbol=<optimized out>, hashcode=<optimized out>) at lookup.cpp:24
#7  find_promise_in_single_environment (symbol=symbol@entry=0x1c52980, rho=rho@entry=0x296e5a38) at lookup.cpp:57
#8  0x00007fffe0b434fd in find_promise_in_environment (symbol=0x1c52980, rho2=rho2@entry=0x33d9a928) at lookup.cpp:78
#9  0x00007fffe0b4b188 in get_full_type_inner (sexp=0x1c52980, rho=0x33d9a928, result=std::vector of length 1, capacity 1 = {...}, visited=std::set with 1 elements = {...}) at sexptypes.cpp:160
#10 0x00007fffe0b4b4e5 in get_full_type (promise=promise@entry=0x33d9b7b8, result=std::vector of length 1, capacity 1 = {...}) at sexptypes.cpp:176
#11 0x00007fffe0b49c09 in create_promise_get_info (context=context@entry=0x3845380, promise=0x33d9b7b8, rho=<optimized out>) at recorder.cpp:303
#12 0x00007fffe0b45089 in promise_created (context=0x3845380, prom=<optimized out>, rho=<optimized out>) at probes.cpp:218

This is the same as all other issues reported.

kondziu commented 6 years ago

[1/1] Executing file: /home/kondziu/workspace/promise-dyntracing-experiment/data/2018-01-25-15-17-33/vignettes/covmat-CovarianceEstimation.R
Loading required package: mvtnorm
Loading required package: RMTstat
Loading required package: grid
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Attaching package: ‘xts’

The following objects are masked from ‘package:dplyr’:

    first, last

Loading required package: fit.models
Loading required package: foreach
Loading required package: PerformanceAnalytics

Attaching package: ‘PerformanceAnalytics’

The following object is masked from ‘package:graphics’:

    legend

Loading required package: rugarch
Loading required package: parallel
DYNTRACE LOG - ERROR - lookup.cpp · get_binding_value(...) · 19 - encountered ACTIVE BINDING frame, missing implementation, confused

PRL-PRG / promisedyntracer

Nested hook execution in package "covmat" #19