segments_selector_congas ERROR

[wyt14]

Hi， I meet the error as below when I run the following code:

code:

filt = segments_selector_congas(x,score="BIC")

Warning message: replacing previous import ‘cli::num_ansi_colors’ by ‘crayon::num_ansi_colors’ when loading ‘easypar’ replacing previous import ‘cli::num_ansi_colors’ by ‘crayon::num_ansi_colors’ when loading ‘easypar’ Error in f(): ! Argument 1 must be a data frame or a named atomic vector. Run rlang::last_trace() to see where the error occurred.

Looks like this is the underlying error location: runs <- easypar::run(FUN = one k, PARAMS = apply(k lambda pairs,1,list)，parallel = parallel, progress bar = FALSE) Fit with k = 2 and lambda = 0.5. [easypar] 1/1 computations returned errors and will be removed. runs NULL

This is the data I used:

x ── [ (R)CONGAS+ ] Bimodal P4 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ── CNA segments (reference: GRCh38) → Input 22 CNA segments, mean ploidy 2.

      | | | |  |  |  | |  |  |  |  |  

     Ploidy:    0     1     2     3     4     5     *   

── Modalities → RNA: 7727 cells with 11101 mapped genes, 8889659 non-zero values. Likelihood: Negative Binomial. → ATAC: not available ! Clusters: not available.

── LOG ──

• 2023-12-21 09:40:26 Created input object.
• 2023-12-21 09:47:01 Filtered s123egments: [0|50|50]
• 2023-12-21 09:54:13 Filtered s123egments: [0|50|50]

Thank you very much!

[Militeee]

Hi @wyt14 unfortunately easypar sometimes hides a lot of information about the actual error, can you provide a minimal input to reproduce the bug?

BW, S.

[wyt14]

tumor_count_test 51 x 20 sparse Matrix of class "dgCMatrix" [[ suppressing 20 column names ‘AAACCTGAGAGAGCTC-1’, ‘AAACCTGAGCCCAACC-1’, ‘AAACCTGAGTACATGA-1’ ... ]]

LINC01346 . . . . . . . . . . . . . . . . . . . . LINC01345 . . . . . . . . . . . . . . . . . . . . LINC02780 . . . . . . . . . . . . . . . . . . . . AL805961.1 . . . . . . . . . . . . . . . . . . . . LINC01777 . . . . . . . . . . . . . . . . . . . . AL355602.1 . . . . . . . . . . . . . . . . . . . . Z98747.1 . . . . . . . . . . . . . . . . . . . . LINC01646 . . . . . . . . . . . . . . . . . . . . AJAP1 . . . . . . . . . . . . . . . . . . . . Z98886.1 . . . . . . . . . . . . . . . . . . . . BX005132.1 . . . . . . . . . . . . . . . . . . . . LINC02781 . . . . . . . . . . . . . . . . . . . . LINC02782 . . . . . . . . . . . . . . . . . . . . AL139823.1 . . . . . . . . . . . . . . . . . . . . Z98259.3 . . . . . . . . . . . . . . . . . . . . Z98259.2 . . . . . . . . . . . . . . . . . . . . Z98259.1 . . . . . . . . . . . . . . . . . . . . AL365255.1 . . . . . . . . . . . . . . . . . . . . NPHP4 . . . . . 1 . . . . . . . . . . . . . . KCNAB2 . . . . . . . . . . . . . . . . . . . . CHD5 . . . . . . . . . . . . . . . . . . . . RPL22 10 4 4 5 13 5 4 12 17 2 12 7 5 6 4 4 14 9 8 8 AL031847.1 . . . . . . . . . . . . . . . . . . . . RNF207 . . . . . . . . . . . . . . . . . . . . ICMT . 1 . . . 1 . . . . . . 1 . . . . . . . LINC00337 . . . . . . . . . . . . . . . . . . . . HES3 . . . . . . . . . . . . . . . . . . . . GPR153 1 . . . . . . 2 . . . . . . . . . . . . ACOT7 . . 2 . . 1 3 . . . 1 1 . 1 . . 2 . 1 . AL031848.2 . . . . . . . . . . . . . . . . . . . . HES2 . . . . . . . . 2 2 . . 2 . . . . . . . ESPN . . 1 . . . . . 1 . . . . . . . . . . . AL031848.1 . . . . . . . . . . . . . . . . . . . .

These are the data I use for my tests. The following are also the underlying errors:

$22 <error/vctrs_error_subscript_oob> Error in pull(): ! Can't extract columns that don't exist. ✖ Column BIC doesn't exist.

Backtrace: ▆

1. ├─tryCatch(parallel:::.workRSOCK, error = function(e) parallel:::.slaveRSOCK)()
2. │ └─parallel:::workLoop(...)
3. │ └─parallel:::workCommand(master)
4. │ ├─base::tryCatch(...)
5. │ │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
6. │ │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
7. │ │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
8. │ ├─base::tryCatch(...)
9. │ │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
   10. │ │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
   11. │ │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
   12. │ ├─base::do.call(msg$data$fun, msg$data$args, quote = TRUE)
   13. │ └─doParallel (local) <fn>(base::quote(<named list>))
   14. │ ├─base::tryCatch(...)
   15. │ │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
   16. │ │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
   17. │ │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
   18. │ └─base::eval(.doSnowGlobals$expr, envir = .doSnowGlobals$exportenv)
   19. │ └─base::eval(.doSnowGlobals$expr, envir = .doSnowGlobals$exportenv)
   20. │ ├─base::do.call(FUN, PARAMS[[i]])
   21. │ └─easypar (local) <fn>(...)
   22. │ └─Rcongas:::fit_congas(...)
   23. │ ├─base::order(model_selection_df %>% pull(!!model_selection))
   24. │ └─model_selection_df %>% pull(!!model_selection)
   25. ├─dplyr::pull(., !!model_selection)
   26. └─dplyr:::pull.data.frame(., !!model_selection)

[wyt14]

Is this the process for handling the expression of data? There shouldn't be any issues, right?

rna = Rcongas::create_congas_tibble(counts = tumor_count,

• modality = 'RNA',
• save_dir=NULL,
• features = features) ℹ Saving temporary counts file Rows: 21552310 Columns: 3
  ── Column specification ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────Delimiter: " " dbl (3): 36601, 7727, 21552310

ℹ Use spec() to retrieve the full column specification for this data. ℹ Specify the column types or set show_col_types = FALSE to quiet this message. Joining with by = join_by(feature_index) Joining with by = join_by(cell_index)

rna = rna %>% mutate(cell = paste0(cell, '-RNA'))

norm_rna = Rcongas:::auto_normalisation_factor(rna) %>% mutate(modality = 'RNA') → Computing library size factors as total counts. → Median digits per factor 4, scaling by 10000. Min. 1st Qu. Median Mean 3rd Qu. Max. 0.4175 0.6508 0.9362 1.1595 1.4574 6.9717 rna = rna %>% filter_known_genes(what='r')

── Ribosomal ℹ n = 186 genes found. MRPL20, RPL22, RPL11, MRPS15, RPS8, RPL5 , ...

── Specials ℹ n = 1 genes found.

all_genes = rna$gene %>% unique mito = all_genes %>% str_starts(pattern = 'MT-') all_genes = setdiff(all_genes, all_genes[mito]) rna = rna %>% dplyr::filter(gene %in% all_genes) rna$value = as.integer(rna$value) rna

A tibble: 20,536,826 × 6

cell value gene chr from to

1 AAACCTGAGAGAGCTC-1-RNA 1 ISG15 chr1 1001137 1013497 2 AAACCTGAGAGAGCTC-1-RNA 2 UBE2J2 chr1 1273725 1273885 3 AAACCTGAGAGAGCTC-1-RNA 1 INTS11 chr1 1324636 1324691 4 AAACCTGAGAGAGCTC-1-RNA 1 AURKAIP1 chr1 1375156 1375495 5 AAACCTGAGAGAGCTC-1-RNA 1 SSU72 chr1 1574583 1574863 6 AAACCTGAGAGAGCTC-1-RNA 1 SKI chr1 2228318 2228319 7 AAACCTGAGAGAGCTC-1-RNA 1 RER1 chr1 2391774 2394005 8 AAACCTGAGAGAGCTC-1-RNA 1 PEX10 chr1 2412529 2412571 9 AAACCTGAGAGAGCTC-1-RNA 1 GPR153 chr1 6261097 6261098 10 AAACCTGAGAGAGCTC-1-RNA 3 PARK7 chr1 7961691 7961969 # ℹ 20,536,816 more rows # ℹ Use `print(n = ...)` to see more rows

[Militeee]

Yea looks like the input is ok, I suspect this is an issue with the underlying python code. What if you run filt = segments_selector_congas(x,score="BIC",what=="nbmix" ) ?

[wyt14]

### Yes, your inference should be correct, something is wrong with the python code. But I ran it with filt = segments_selector_congas(x,score="BIC",what=="nbmix") and still got an error. The error report is as follows:

── Fit with k = 1 and lambda = 0.5.

── Fit with k = 1 and lambda = 0.5. sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.)

sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) ── Fit with k = 2 and lambda = 0.5. ── Fit with k = 2 and lambda = 0.5. ── Fit with k = 2 and lambda = 0.5. sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.)

sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.)

── Fit with k = 2 and lambda = 0.5.

── Fit with k = 2 and lambda = 0.5. ── Fit with k = 2 and lambda = 0.5. ── Fit with k = 2 and lambda = 0.5.

sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.)

sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.)

── Fit with k = 3 and lambda = 0.5. ── Fit with k = 3 and lambda = 0.5. sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.)

── Fit with k = 2 and lambda = 0.5. ── Fit with k = 3 and lambda = 0.5.

sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.)

sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.)

sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) ── Fit with k = 2 and lambda = 0.5. ── Fit with k = 2 and lambda = 0.5.

── Fit with k = 2 and lambda = 0.5.

── Fit with k = 2 and lambda = 0.5.

── Fit with k = 2 and lambda = 0.5. ── Fit with k = 3 and lambda = 0.5. ── Fit with k = 2 and lambda = 0.5.

── Fit with k = 2 and lambda = 0.5. sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.)

── Fit with k = 3 and lambda = 0.5. ── Fit with k = 3 and lambda = 0.5. ── Fit with k = 2 and lambda = 0.5. [easypar] 3/3 computations returned errors and will be removed.

── Fit with k = 2 and lambda = 0.5.

[easypar] 3/3 computations returned errors and will be removed.

── Fit with k = 2 and lambda = 0.5. sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) ── Fit with k = 3 and lambda = 0.5.

sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) ── Fit with k = 2 and lambda = 0.5.

sys:1: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /croot/pytorch_1675190298929/work/torch/csrc/utils/tensor_numpy.cpp:172.) ── Fit with k = 2 and lambda = 0.5.

── Fit with k = 3 and lambda = 0.5. [easypar] 3/3 computations returned errors and will be removed.

── Fit with k = 3 and lambda = 0.5. [easypar] 3/3 computations returned errors and will be removed.

[easypar] 3/3 computations returned errors and will be removed.

── Fit with k = 3 and lambda = 0.5. ── Fit with k = 3 and lambda = 0.5.

── Fit with k = 2 and lambda = 0.5. [easypar] 3/3 computations returned errors and will be removed.

── Fit with k = 3 and lambda = 0.5.

── Fit with k = 2 and lambda = 0.5. ── Fit with k = 3 and lambda = 0.5. [easypar] 3/3 computations returned errors and will be removed.

[easypar] 3/3 computations returned errors and will be removed.

── (R)CONGAS+ fits completed in 10.5s. ── ── (R)CONGAS+ fits completed in 10.9s. ── ── (R)CONGAS+ fits completed in 10.6s. ──

Error in f(): ! Argument 1 must be a data frame or a named atomic vector. Run rlang::last_trace() to see where the error occurred.

[wyt14]

In addition, I would like to ask whether rcongas invokes python version 2 or 3, and whether it is related to python version (I used Python 3.10) or what part of the code should I change. Thank you~

[lucreziaPatruno]

Hi @wyt14, I haven't been able to reproduce your issue unfortunately. It's possible that something went wrong in the reticulate setup and CONGAS+ inference doesn't start, hence why you get that error. Can you please try to load congas through reticulate like this:

library(reticulate)
cg = reticulate::import("congas")

and please let me know whether you have any issue or if you are able to load it without any error.

BW Lucrezia

[caravagn]

Hi @wyt14 - thanks for trying out CONGAS+. Based on the errors, this might also be a problem with the current configuration of your packages. It would be important for us to have a minimal dataset and code to exaclt reproduce your error.

Of course we do understand the importance of maintaining your data confidential. Was this an issue, it would be possible for us also to work with dummy data.

Can you provide it?

[wyt14]

Hi @wyt14, I haven't been able to reproduce your issue unfortunately. It's possible that something went wrong in the reticulate setup and CONGAS+ inference doesn't start, hence why you get that error. Can you please try to load congas through reticulate like this:

library(reticulate)
cg = reticulate::import("congas")

and please let me know whether you have any issue or if you are able to load it without any error.

BW Lucrezia

First of all, thanks to Lucrezia, I ran the code you mentioned without any error message.

[wyt14]

Hi @wyt14 - thanks for trying out CONGAS+. Based on the errors, this might also be a problem with the current configuration of your packages. It would be important for us to have a minimal dataset and code to exaclt reproduce your error.

Of course we do understand the importance of maintaining your data confidential. Was this an issue, it would be possible for us also to work with dummy data.

Can you provide it?

Thanks to caravagn. Below is the running code and the data used. Thank you very much~ rcongas.zip

[Militeee]

Hi @wyt14 ,

Sorry for the late response but Holiday period slowed everything here, I have committed a fix, can you try reinstalling the tool and rerunning the segment selector?

@lucreziaPatruno PyTorch does not like easypar in parallel mode, we have to make sure at a certain point we are never using them together

[wyt14]

@Militeee @lucreziaPatruno Thank you very much for your help, I have successfully run. In addition, there is a small question, is there a parameter in fit_congas function to set the number of cpu cores to use? Or does rcongas have a principle to follow when using the number of kernels?

[Militeee]

So the decision on how to use the resources is generally up to PyTorch, if you want to modify the number of threads used, you can run, before the fit:

torch = reticulate::import("torch")
torch$set_num_threads(number_of_threads)

Take also a look at that for the other various options.

I close the issue now, feel free to open a new one if you still need help.

Cheers, S.