Problem with initializeScenic()

[Shaiov]

Hello, thank you for the great tool!

I used SCENIC many times, running it with macOS and it worked very well and had no probloms. After updating my macOS to the new Big Sur update i am trying to run SCENIC again but i get this warning in initializeScenic():

Motif databases selected: mm9-500bp-upstream-7species.mc9nr.feather mm9-tss-centered-10kb-7species.mc9nr.feather Warning messages: 1: In RcisTarget::importRankings(dbFilePath, columns = randomCol) : The following columns are missing from the database: 2: In RcisTarget::importRankings(dbFilePath, columns = randomCol) : The following columns are missing from the database:

Is it possible that this problem is due to the update? is there something to do?

thank you very much!

[mauriziopaul]

Hi! I am having some similar issues with the hgnc annotations on macOS 11.2.1. I appreciate any suggestions you might have!

> scenicOptions <- initializeScenic(org="hgnc", dbDir=".", nCores=2)

Motif databases selected: 
  hg19-500bp-upstream-7species.mc9nr.feather 
  hg19-tss-centered-10kb-7species.mc9nr.feather
Warning messages:
1: In RcisTarget::importRankings(dbFilePath, columns = randomCol) :
  The following columns are missing from the database: 
2: In RcisTarget::importRankings(dbFilePath, columns = randomCol) :
  The following columns are missing from the database: 

> sessionInfo()
R version 4.0.4 (2021-02-15)
Platform: x86_64-apple-darwin20.3.0 (64-bit)
Running under: macOS Big Sur 10.16

Matrix products: default
LAPACK: /usr/local/Cellar/r/4.0.4/lib/R/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] grid      parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] SCopeLoomR_0.10.4    SCENIC_1.2.4         doRNG_1.8.2          doMC_1.3.7           iterators_1.0.13     foreach_1.5.1        Rtsne_0.15          
 [8] R2HTML_2.3.2         ComplexHeatmap_2.6.2 NMF_0.23.0           synchronicity_1.3.5  bigmemory_4.5.36     Biobase_2.50.0       BiocGenerics_0.36.0 
[15] cluster_2.1.1        rngtools_1.5         pkgmaker_0.32.2      registry_0.5-1       DT_0.17              rbokeh_0.5.1         mixtools_1.2.0      
[22] zoo_1.8-8            GENIE3_1.12.0        RcisTarget_1.10.0    AUCell_1.12.0       

[9daiendo]

I'm suffering from the same error. If anyone has a solution, please let me know.

> hg38_dbs <- list('500bp'= 'hg38__refseq-r80__500bp_up_and_100bp_down_tss.mc9nr.feather',  '10kb' = 'hg38__refseq-r80__10kb_up_and_down_tss.mc9nr.feather')
> db_mcVersion <- 'v9'
> db_path <- '(my working directory)/cisTarget_databases' 
> scenicOptions <- initializeScenic(org = 'hgnc', dbDir = db_path, dbs = hg38_dbs, datasetTitle='SCENIC', nCores=10)

Motif databases selected: 
  hg38__refseq-r80__500bp_up_and_100bp_down_tss.mc9nr.feather 
  hg38__refseq-r80__10kb_up_and_down_tss.mc9nr.feather
 Warning messages: 
1:  RcisTarget::importRankings(dbFilePath, columns = randomCol) : 
  The following columns are missing from the database: 
2:  RcisTarget::importRankings(dbFilePath, columns = randomCol) : 
  The following columns are missing from the database: 

R version 4.0.3 (2020-10-10) Platform: x86_64-apple-darwin17.0 (64-bit) Running under: macOS Big Sur 10.16

[9daiendo]

I get an error message, but scenicOptions was created without any problem. I was able to complete the flow beyond that. It seems that I can ignore that error message.

[AAA-3]

I also had the same error:

Motif databases selected: 
  mm9-500bp-upstream-7species.mc9nr.feather 
  mm9-tss-centered-10kb-7species.mc9nr.feather
Warning messages:
1: In RcisTarget::importRankings(dbFilePath, columns = randomCol) :
  The following columns are missing from the database: 
2: In RcisTarget::importRankings(dbFilePath, columns = randomCol) :
  The following columns are missing from the database: 
3: In RcisTarget::importRankings(dbFile, columns = rnktype) :
  The following columns are missing from the database: features
4: In RcisTarget::importRankings(dbFile, columns = rnktype) :
  The following columns are missing from the database: features

This seems to be expected since the tutorial on SCENIC also shows this error as output...

[sofiapuvogelvittini]

Hello! Does anybody could solve the error? I am with the same problem: Motif databases selected: hg19-500bp-upstream-7species.mc9nr.feather hg19-tss-centered-10kb-7species.mc9nr.feather

[1] "there is no package called ‘arrow’" [1] "there is no package called ‘arrow’"

Warning message in initializeScenic(org = "hgnc", dbDir = dbDir, nCores = 10): “It was not possible to load the following databses; check whether they are downloaded correctly: hg19-500bp-upstream-7species.mc9nr.feather hg19-tss-centered-10kb-7species.mc9nr.feather” Warning message in RcisTarget::importRankings(dbFile, columns = rnktype): “The following columns are missing from the database: features” Warning message in RcisTarget::importRankings(dbFile, columns = rnktype): “The following columns are missing from the database: features”

Thanks for your help

[Coke-Zhang]

Hello! Does anybody could solve the error? I am with the same problem: Motif databases selected: hg19-500bp-upstream-7species.mc9nr.feather hg19-tss-centered-10kb-7species.mc9nr.feather

[1] "there is no package called ‘arrow’" [1] "there is no package called ‘arrow’"

Warning message in initializeScenic(org = "hgnc", dbDir = dbDir, nCores = 10): “It was not possible to load the following databses; check whether they are downloaded correctly: hg19-500bp-upstream-7species.mc9nr.feather hg19-tss-centered-10kb-7species.mc9nr.feather” Warning message in RcisTarget::importRankings(dbFile, columns = rnktype): “The following columns are missing from the database: features” Warning message in RcisTarget::importRankings(dbFile, columns = rnktype): “The following columns are missing from the database: features”

Thanks for your help

Just need install arrow. http://arrow.apache.org/docs/r/articles/install.html. install.packages("arrow") . Or have error arrow::install_arrow(verbose = TRUE)

[Abigail575]

Hi, I have the same problem, but after installing and loading "arrow", my R session just keeps aborting...can anyone help please? Thanks!

[kmshort]

I get this error too and arrow is installed fine. The feather files load fine using read_feather(), and the features column is there, and gene name column headers are there. A scenicOptions object is still generated, but I'm not sure of what'll come out at the end..

[WRui]

Hello! Does anybody could solve the error? I am with the same problem: Motif databases selected: hg19-500bp-upstream-7species.mc9nr.feather hg19-tss-centered-10kb-7species.mc9nr.feather

[1] "there is no package called ‘arrow’" [1] "there is no package called ‘arrow’"

Warning message in initializeScenic(org = "hgnc", dbDir = dbDir, nCores = 10): “It was not possible to load the following databses; check whether they are downloaded correctly: hg19-500bp-upstream-7species.mc9nr.feather hg19-tss-centered-10kb-7species.mc9nr.feather” Warning message in RcisTarget::importRankings(dbFile, columns = rnktype): “The following columns are missing from the database: features” Warning message in RcisTarget::importRankings(dbFile, columns = rnktype): “The following columns are missing from the database: features”

Thanks for your help

you can run the following code : install.package("arrow") library(arrow) install_arrow(verbose=TRUE)

[Abigail575]

Hi! Yes, I asked someone with Linux to download them for me. Then when I imported them, it was fine. Maybe you are using Windows? Here is the link https://resources.aertslab.org/cistarget/help.html.

[Nina-Lydia]

Hello! Has anyone managed to find a fix for this? I get the same error; I have downloaded the databases properly I think and checked that the download was correct and complete, but still getting the error. Arrow is there too. I have also checked the feather files and the columns reported as missing in the error are there. I can just ignore the error and continue with the pipeline, as the scenicOptions object is created, but then the output object is basically empty. Thanks in advance !

Motif databases selected: hg19-500bp-upstream-7species.mc9nr.feather hg19-tss-centered-10kb-7species.mc9nr.feather Warning in RcisTarget::importRankings(dbFilePath, columns = randomCol) : The following columns are missing from the database: Warning in RcisTarget::importRankings(dbFilePath, columns = randomCol) : The following columns are missing from the database: Warning in RcisTarget::importRankings(dbFile, columns = rnktype) : The following columns are missing from the database: features Warning in RcisTarget::importRankings(dbFile, columns = rnktype) : The following columns are missing from the database: features

[s-aibar]

These errors seem to be related to the R version... We had to change the use of feather package by arrow.

If you keep having the issue with the latest version of R and related packages (e.g. RcisTarget), please let us know.

(I'll close the issue, but feel free to re-open or post a new one with updated information if the problem persists...)

[stevehxf]

Hi,

I used to use 3.x R and I did not see any problem. But we recently updated R to 4.2.1. I got errors when I ran initializeScenic.

Could anyone help please?

scenicOptions <- initializeScenic(org="hgnc",

• nCores=10,
• dbDir=mydbDIR,
• dbs = mydbs,
• datasetTitle = "HNSCC") Motif databases selected: /hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.feather /hg38refseq-r8010kb_up_and_down_tss.mc9nr.feather [1] "The index column 'features' is not available in the file." [1] "The index column 'features' is not available in the file." Using the column 'A1BG' as feature index for the ranking database. Error in vec_as_location2_result(): ! Can't extract columns that don't exist. ✖ Column features doesn't exist. Run rlang::last_error() to see where the error occurred. Warning messages: 1: In initializeScenic(org = "hgnc", nCores = 10, dbDir = mydbDIR, : It was not possible to load the following databses; check whether they are downloaded correctly: /hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.feather /hg38refseq-r8010kb_up_and_down_tss.mc9nr.feather 2: In RcisTarget::importRankings(dbFile, columns = rnktype) : The following columns are missing from the database: features

[P-G-Park]

same problem here.

dbPath <- './raw_data/attempt_2/221011_scenic/hg38_10.feather' SCENIC::dbLoadingAttempt(dbPath) [1] "The index column 'features' is not available in the file." [1] FALSE

[WangYuzee]

你好，

我曾经使用 3.x R 并没有发现任何问题。但我们最近将 R 更新为 4.2.1。运行 initializeScenic 时出现错误。

有人可以帮忙吗？

SceneOptions <- initializeScenic(org="hgnc",

•                             nCores=10,

•                             dbDir=mydbDIR, 

•                             dbs = mydbs,

•                             datasetTitle = "HNSCC")

选择的主题数据库： /hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.feather /hg38refseq-r8010kb_up_and_down_tss.mc9nr.feather [1]“文件中没有索引列‘特征’。” [1] “文件中没有索引列‘特征’。” 使用列“A1BG”作为排名数据库的特征索引。 错误： ！无法提取不存在的列。使用列“A1BG”作为排名数据库的特征索引。vec_as_location2_result() ✖列features不存在。 运行rlang::last_error()以查看错误发生的位置。 警告信息： 1: In initializeScenic(org = "hgnc", nCores = 10, dbDir = mydbDIR, : 无法加载以下数据库；检查它们是否下载正确： /hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.feather /hg38refseq -r8010kb_up_and_down_tss.mc9nr.feather 2：在 RcisTarget::importRankings(dbFile, columns = rnktype) 中： 数据库中缺少以下列：特征

Hi,I meet the similar problem, have you solve it?

[WangYuzee]

scenicOptions <- initializeScenic(org="hgnc", dbDir="cisTarget_databases/", dbs = dbs,nCores=4) Motif databases selected: hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.genes_vs_motifs.rankings.feather hg38refseq-r8010kb_up_and_down_tss.mc9nr.genes_vs_motifs.rankings.feather [1] "The index column 'features' is not available in the file." [1] "The index column 'features' is not available in the file." Using the column 'A1BG' as feature index for the ranking database. Error in dplyr::pull(): ! Can't extract columns that don't exist. ✖ Column features doesn't exist. Run rlang::last_error() to see where the error occurred. Warning messages: 1: In initializeScenic(org = "hgnc", dbDir = "cisTarget_databases/", : It was not possible to load the following databses; check whether they are downloaded correctly: hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.genes_vs_motifs.rankings.feather hg38refseq-r8010kb_up_and_down_tss.mc9nr.genes_vs_motifs.rankings.feather 2: In RcisTarget::importRankings(dbFile, columns = rnktype) : The following columns are missing from the database: features

Have someone meet this problem?

[AClab-sgarcia]

scenicOptions <- initializeScenic(org="hgnc", dbDir="cisTarget_databases/", dbs = dbs,nCores=4) Motif databases selected: hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.genes_vs_motifs.rankings.feather hg38refseq-r8010kb_up_and_down_tss.mc9nr.genes_vs_motifs.rankings.feather [1] "The index column 'features' is not available in the file." [1] "The index column 'features' is not available in the file." Using the column 'A1BG' as feature index for the ranking database. Error in dplyr::pull(): ! Can't extract columns that don't exist. ✖ Column features doesn't exist. Run rlang::last_error() to see where the error occurred. Warning messages: 1: In initializeScenic(org = "hgnc", dbDir = "cisTarget_databases/", : It was not possible to load the following databses; check whether they are downloaded correctly: hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.genes_vs_motifs.rankings.feather hg38refseq-r8010kb_up_and_down_tss.mc9nr.genes_vs_motifs.rankings.feather 2: In RcisTarget::importRankings(dbFile, columns = rnktype) : The following columns are missing from the database: features

Have someone meet this problem?

I'm currently having the same problem.

[ybaeus]

scenicOptions <- initializeScenic(org="hgnc", dbDir="cisTarget_databases/", dbs = dbs,nCores=4) Motif databases selected: hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.genes_vs_motifs.rankings.feather hg38refseq-r8010kb_up_and_down_tss.mc9nr.genes_vs_motifs.rankings.feather [1] "The index column 'features' is not available in the file." [1] "The index column 'features' is not available in the file." Using the column 'A1BG' as feature index for the ranking database. Error in dplyr::pull(): ! Can't extract columns that don't exist. ✖ Column features doesn't exist. Run rlang::last_error() to see where the error occurred. Warning messages: 1: In initializeScenic(org = "hgnc", dbDir = "cisTarget_databases/", : It was not possible to load the following databses; check whether they are downloaded correctly: hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.genes_vs_motifs.rankings.feather hg38refseq-r8010kb_up_and_down_tss.mc9nr.genes_vs_motifs.rankings.feather 2: In RcisTarget::importRankings(dbFile, columns = rnktype) : The following columns are missing from the database: features

Have someone meet this problem?

I faced the exact same issue. Did you solve the problem?

[WT215]

scenicOptions <- initializeScenic(org="hgnc", dbDir="cisTarget_databases/", dbs = dbs,nCores=4) Motif databases selected: hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.genes_vs_motifs.rankings.feather hg38refseq-r8010kb_up_and_down_tss.mc9nr.genes_vs_motifs.rankings.feather [1] "The index column 'features' is not available in the file." [1] "The index column 'features' is not available in the file." Using the column 'A1BG' as feature index for the ranking database. Error in dplyr::pull(): ! Can't extract columns that don't exist. ✖ Column features doesn't exist. Run rlang::last_error() to see where the error occurred. Warning messages: 1: In initializeScenic(org = "hgnc", dbDir = "cisTarget_databases/", : It was not possible to load the following databses; check whether they are downloaded correctly: hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.genes_vs_motifs.rankings.feather hg38refseq-r8010kb_up_and_down_tss.mc9nr.genes_vs_motifs.rankings.feather 2: In RcisTarget::importRankings(dbFile, columns = rnktype) : The following columns are missing from the database: features Have someone meet this problem?

I faced the exact same issue. Did you solve the problem?

I ended up uisng pyscenic, which is reasonably fast than R version.

[ribaribaworking]

The same problem here. Checked the feather object and it has no "feature" column.

Motif databases selected: hg38_500bp_up_100bp_down_full_tx_v10_clust.genes_vs_motifs.rankings.feather hg38_10kbp_up_10kbp_down_full_tx_v10_clust.genes_vs_motifs.rankings.feather [1] "The index column 'features' is not available in the file." [1] "The index column 'features' is not available in the file." Using the column 'motifs' as feature index for the ranking database. Error in dplyr::pull(): ! Can't extract columns that don't exist. ✖ Column features doesn't exist

[HyeonwooLa]

try trace(checkAnnots, edit = T) and edit the fifth line from rnktype = "features" to rnktype = "motifs"

[AlexBui26]

Hi! I got the same issue running on R4.2.1

scenicOptions <- initializeScenic(org="hgnc", dbDir=dbDir,dbs=dbName, nCores=10)
Motif databases selected:
hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.genes_vs_motifs.rankings.feather
hg38refseq-r8010kb_up_and_down_tss.mc9nr.genes_vs_motifs.rankings.feather
[1] "The index column 'features' is not available in the file."
[1] "The index column 'features' is not available in the file."
Using the column 'A1BG' as feature index for the ranking database.
Error in dplyr::pull():
! Can't extract columns that don't exist.
✖ Column features doesn't exist.
Run rlang::last_trace() to see where the error occurred.
Warning messages:
1: In initializeScenic(org = "hgnc", dbDir = dbDir, dbs = dbName, nCores = 10) :
It was not possible to load the following databses; check whether they are downloaded correctly:
hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.genes_vs_motifs.rankings.feather
hg38refseq-r8010kb_up_and_down_tss.mc9nr.genes_vs_motifs.rankings.feather
2: In RcisTarget::importRankings(dbFile, columns = rnktype) :
The following columns are missing from the database: features

Is there a way to fix this in R now? Or switching to pySCENIC is the only option? Thank you!!

[AlexBui26]

These errors seem to be related to the R version... We had to change the use of feather package by arrow.

If you keep having the issue with the latest version of R and related packages (e.g. RcisTarget), please let us know.

(I'll close the issue, but feel free to re-open or post a new one with updated information if the problem persists...)

Did you mean to change the source code of SCENIC? I am using SCENIC 1.3.1. Is a updated version to correct this issue available now? Thanks!

[JWDu0122]

try trace(checkAnnots, edit = T) and edit the fifth line from rnktype = "features" to rnktype = "motifs"

it's run, thanks!