pycox error

[zhangkaicr]

Hello, I am a doctor of oncology from Tianjin Medical University Cancer Hospital. I recently learned about your mlr3prob package. Thank you very much for developing this package, which allows us to combine many machine-learning algorithms to run batch analysis. I built a virtual environment of mlr3 in the server using conda, and used conda to install the latest versions of pyhont, tensflow and pycox in the virtual environment. After running the code from your official website below, I still get an error. I don’t know what the problem is. Thank you very much for your help!

Your tutorial URL https://mlr3gallery.mlr-org.com/posts/2021-04-20-survival-networks/

library(mlr3verse)
library(reticulate)
library(mlr3proba)
library(survivalmodels)

use_condaenv(condaenv="mlr3", require = TRUE)

set_seed(1234)

library(mlr3)
library(mlr3proba)

## get the `whas` task from mlr3proba
whas <- tsk("whas")

## create our own task from the rats dataset
rats_data <- survival::rats
## convert characters to factors
rats_data$sex <- factor(rats_data$sex, levels = c("f", "m"))
rats <- TaskSurv$new("rats", rats_data, time = "time", event = "status")

## combine in list
tasks <- list(whas, rats)

library(paradox)
search_space = ps(
  # p_dbl for numeric valued parameters
  dropout = p_dbl(lower = 0, upper = 1),
  weight_decay = p_dbl(lower = 0, upper = 0.5),
  learning_rate = p_dbl(lower = 0, upper = 1),

  # p_int for integer valued parameters
  nodes = p_int(lower = 1, upper = 32),
  k = p_int(lower = 1, upper = 4)
)

search_space$trafo = function(x, param_set) {
  x$num_nodes = rep(x$nodes, x$k)
  x$nodes = x$k = NULL
  return(x)
}

library(mlr3tuning)

create_autotuner <- function(learner) {
  AutoTuner$new(
    learner = learner,
    search_space = search_space,
    resampling = rsmp("holdout"),
    measure = msr("surv.cindex"),
    terminator = trm("evals", n_evals = 2),
    tuner = tnr("random_search"))
}

library(mlr3extralearners)

## load learners
learners <- lrns(paste0("surv.", c("coxtime", "deephit", "deepsurv", 
                                   "loghaz", "pchazard")),
                 frac = 0.3, early_stopping = TRUE, epochs = 10,
                 optimizer = "adam"
)

# apply our function
learners <- lapply(learners, create_autotuner)

library(mlr3pipelines)

create_pipeops <- function(learner) {
  po("encode") %>>% po("scale") %>>% po("learner", learner)
}

# apply our function
learners <- lapply(learners, create_pipeops)

## select holdout as the resampling strategy
resampling <- rsmp("cv", folds = 3)

## add KM and CPH
learners <- c(learners, lrns(c("surv.kaplan", "surv.coxph")))
design <- benchmark_grid(tasks, learners, resampling)
bm <- benchmark(design)`

The error message is as follows

INFO  [21:20:54.966] [bbotk]  0.6683808    0.1748101      0.101513     5 4          <list[8]> <list[4]>   0.5761905
INFO  [21:20:57.412] [mlr3] Applying learner 'surv.kaplan' on task 'rats' (iter 1/3)
INFO  [21:20:57.477] [mlr3] Applying learner 'surv.kaplan' on task 'rats' (iter 2/3)
INFO  [21:20:57.539] [mlr3] Applying learner 'surv.kaplan' on task 'rats' (iter 3/3)
INFO  [21:20:57.597] [mlr3] Applying learner 'surv.coxph' on task 'rats' (iter 1/3)
INFO  [21:20:57.655] [mlr3] Applying learner 'surv.coxph' on task 'rats' (iter 2/3)
INFO  [21:20:57.714] [mlr3] Applying learner 'surv.coxph' on task 'rats' (iter 3/3)
Error in py_call_impl(callable, call_args$unnamed, call_args$named) : 
  AttributeError: 'Series' object has no attribute 'iteritems'
Run `reticulate::py_last_error()` for details.

[RaphaelS1]

The error is in the package pycox, I think you'll need to install a different version of that package. I am aware that there are an increasing number of bugs in pycox but unfortunately there's a limit to what I can do to fix them

[bblodfon]

@RaphaelS1 Maybe this should be transferred to pycox or survivalmodels repo?

[zhangkaicr]

The error is in the package pycox, I think you'll need to install a different version of that package. I am aware that there are an increasing number of bugs in pycox but unfortunately there's a limit to what I can do to fix them

Thank you for your answer. Your R package is the most complete package that contains the machine learning survival algorithm in the R language that I know of. Can you share the version information of all dependent packages when you deploy the test code?

[bblodfon]

@jemus42 have you managed to get pycox to run? Is it easy to share (if not too trouble) the package versions that make it work for @zhangkaicr?

[jemus42]

Unfortunately I never quite got there (this year, it kind of worked last year...) -- at the end I had basically tried a grid search across different python versions, assuming that would help, but... not really :/

This error is what I got for coxtime under python versions 3.8 through 3.10, there's a chance 3.7 might work? Removing coxtime and using only the others might be worth a try as well, that was the most error-prone learner in my experience.

[RaphaelS1]

@RaphaelS1 Maybe this should be transferred to pycox or survivalmodels repo?

@bblodfon Definitely not survivalmodels, it would be good if someone reported these bugs (and there's a few) in pycox but also just reimplementing everything is definitely the best way forward (if only I had the time)

Thank you for your answer. Your R package is the most complete package that contains the machine learning survival algorithm in the R language that I know of. Can you share the version information of all dependent packages when you deploy the test code?

@zhangkaicr I really wish I had saved that information but I didn't, poor open source practice!

[bblodfon]

@zhangkaicr please post this issue in https://github.com/havakv/pycox, there is nothing more we can do now for this here.