search

epigen / RnBeads

Git working Repo synced to the Bioconductor: http://bioconductor.org/packages/devel/bioc/html/RnBeads.html
https://rnbeads.org/
7 stars 8 forks source link

Infinium MethylationEPIC v2.0 Kit #46

Open jozsefkun opened 9 months ago

jozsefkun commented 9 months ago

Hi, Illumina discontinued the earlier version of the EPIC array. Can you suggest a way to use RnBeads with EPIC v2.0 data?

bariskalem commented 9 months ago

Hi, I'm implementing EPIC v2 compatibility for RnBeads. The package will soon be officially updated.

jozsefkun commented 9 months ago

Many thanks!

haiqingli commented 9 months ago

The current release cannot generate correct QC report and get wrong filter on type II probes (""unreliable measurements") on EPIC v2.0 data. Hope the new release can fix those. BTW, when will the new release with v2.0 support be available? Thanks.

bariskalem commented 8 months ago

Hi @haiqingli , you can expect EPIC v2 support in RnBeads sometime next month. We will also publish an updated annotation package.

bariskalem commented 7 months ago

Dear @jozsefkun and @haiqingli,

You can use the links below for a preview version of RnBeads with EPICv2 compatibility. You need to install the updated RnBeads.hg38 package and the RnBeads package.

We would greatly appreciate your feedback after you've tried this version of RnBeads!

EPICv2 compatible RnBeads: https://github.com/epigen/RnBeads/tree/feature/EPICv2 Annotation Package: https://icbb-share.s3.eu-central-1.amazonaws.com/RnBeads/EPICv2/RnBeads_hg38_preview/RnBeads.hg38_1.34.0.tar.gz Instructions: https://icbb-share.s3.eu-central-1.amazonaws.com/RnBeads/EPICv2/RnBeads_hg38_preview/instructions.txt Install script: https://icbb-share.s3.eu-central-1.amazonaws.com/RnBeads/EPICv2/RnBeads_hg38_preview/install.R

Please keep in mind that this version of RnBeads is intended only to test EPICv2 data and is not tested with the previous Illumina platforms. Implementation for full compatibility is in progress.

jozsefkun commented 7 months ago

@bariskalem Thank you for letting us know. Will give it a try in the coming weeks.

VivianBailey commented 6 months ago

issue#1: I have no problem running "rnb.run.preprocessing". But if I break it down to "rnb.execute.normalization", it generated error "Error in rnb.get.annotation("controlsEPICv2") :unsupported annotation type" and the filtering step before "rnb.execute.greedycut" generated an empty file. issue#2: "rnb.run.import" generated error "Error in h(simpleError(msg, call)) : error in evaluating the argument 'x' in selecting a method for function 'grepl': $ operator is invalid for atomic vectors". issue#3: the RnBeadRawSet from "rnb.execute.import" doesn't have rownames, it causes error (invalid value for probeID; missing in dataset) when run "rnb.plot.snp.barplot". issue#4: "rnb.run.qc" generated error "Error in .local(.Object, ...) : directory cannot be created".

VivianBailey commented 6 months ago

issue#5: plotting issue running "rnb.plot.locus.profile(rnb,chrom,start,end,grps=sample.grouping)" 2023-12-14 14:55:12 Inf WARNING could not create CpG Island track: Unknown track: cpgIslandExt Warning messages: 1: Ensembl will soon enforce the use of https. Ensure the 'host' argument includes "https://" 2: In .local(x, ...) : 'track' parameter is deprecated now you go by the 'table' instead Use ucscTables(genome, track) to retrieve the list of tables for a track 3: 'memory.size()' is no longer supported

bariskalem commented 5 months ago

Hi everyone,

we now have an updated annotation package that can be downloaded by this link: https://icbb-share.s3.eu-central-1.amazonaws.com/RnBeads/EPICv2/RnBeads_hg38_preview/RnBeads.hg38_1.35.0.tar.gz

RnBeads also received an update so it supports the annotation package, make sure to update RnBeads as well: https://github.com/epigen/RnBeads/tree/feature/EPICv2

bariskalem commented 5 months ago

Hi @VivianBailey ,

Thank you for informing us about these issues!

1st issue:

Should be addressed by this new update.

2nd issue:

The data.source parameter of the rnb.run.importfunction needs to be created as a list for your case. Below is an example of how that should look like. Make sure the data.dir identifier is there. data.source <- list(data.dir = idat.dir, sample.annotation = sample.annotation)

3rd issue:

Make sure the SNP probe ID you are giving is a IlmnID. Example: "rs213028_BC11”. I was able to run this code without an issue: 

...
rnb.set <- rnb.execute.import(data.source = data.source, data.type = "idat.dir")
rnb.plot.snp.barplot(rnb.set,”rs213028_BC11")

4th issue:

I was not able to reproduce this issue. My guess is that your script is trying to create a directory that already exists. If you provide more information regarding your analysis script we could try once again to solve your problem.

5th issue:

I suspect this issue to be one that is unrelated to the EPICv2 compatibility update. However I’ve still included a minor fix that I think should help. Please let me know if it works for your case.

I hope these suggestions help. Again, thank you for your feedback!

VivianBailey commented 5 months ago

3rd issue: here is my code rnb.set<-rnb.execute.import(data.source=data.source,data.type="infinium.idat.dir") rnb.plot.snp.heatmap(rnb)#this works and I can see the probe ID e.g. rs213028_BC11 The following does not have any error or any plot rnb.plot.snp.barplot(rnb.set,"rs213028_BC11") The following shows error "invalid value for probeID; missing in dataset" rnb.plot.snp.barplot(rnb.set,samples(rnb.set)[1],"rs213028_BC11") I also tried data.type="idat,dir" instead of "infinium.idat.dir", same issue

VivianBailey commented 5 months ago

4th issue: "rnb.run.qc" works in CNV estimation using "rnb.run.qc(rnb.set.unfiltered,dir.reports=report.dir)" but the Sex prediction plot wasn't created. I tried the following two options.

1.rnb.options(qc.cnv.refbased=TRUE)#default setting,CNV estimation module with reference-based approach

2.rnb.options(qc.cnv=TRUE)#CNV estimation module with reference-free approach

when I ran option#2, there is a warning "CNV calling currently not supported due to dependency issues". I am unable to identify the dependency.

VivianBailey commented 5 months ago

6th question: how to get and RnBset object after rnb.execute.greedycut? The example uses rnb.execute.greedycut(rnb.set.filtered)$dataset,but I only saw info, statistics, iteration, sites and samples.

VivianBailey commented 5 months ago

7th issue: Error in log10(lolaRes[["qValue"]]) : non-numeric argument to mathematical function when running lolaBarPlot enrich.lola<-performLolaEnrichment.diffMeth(rnb.set,diffmeth,lolaDirs[["hg38"]]) enrich.table.lola<-enrich.lola$region[[comparison]][["promoters"]] lolaBarPlot(enrich.lola$lolaDb,enrich.table.lola,scoreCol="oddsRatio",orderCol="maxRnk",pvalCut=0.05)

mhack02 commented 5 months ago

Hello, I installed the updated version of RnBeads and of RnBeads.hg38. Now, I want to create a new RnBSet Object. I am using EpicV2 probes and the function:

rnb.options(filtering.sex.chromosomes.removal = TRUE, identifiers.column = "barcode", import.idat.platform = "auto") data.source <- c(idat.dir, sample.annotation) result <- rnb.run.import(data.source=data.source, data.type="idat.dir", dir.reports=report.dir) rnb.set <- rnb.execute.import(data.source=data.source, data.type="idat.dir") rnb.set <- result$rnb.set

The rnb.set only contains 866895 CpGs like the rnbset created with 850k data. So the 935k probe was not detected correctly. What should I change? I can't find any other other suiting options than "auto". Any suggestions?

Thank you! Marcel

bariskalem commented 5 months ago

can't find any other other suiting options than "auto"

Hi Marcel, Can you try setting import.idat.platform option to "probesEPICv2"?

mhack02 commented 5 months ago

Hi @bariskalem, thank you for the quick reply! I tried setting import.idat.platform option to "probesEPICv2", but it is an invalid value for this option. Is there another way to load a RnBeadRawSet with 935k data?

bariskalem commented 5 months ago

Hi @mhack02, rnb.options(import.idat.platform = "probesEPICv2") If the line above throws an error, you have not installed the updated RnBeads package with EPICv2 support in your R environment. Please install this build of RnBeads: https://github.com/epigen/RnBeads/tree/feature/EPICv2

LucianoCalzari commented 5 months ago

Hi everyone, thanks for addressing this discussion as I'm also having difficulties with the EPIC V2 analysis. I'm also unable to resolve the RnBeads error message:

rnb.options(filtering.sex.chromosomes.removal = FALSE, identifiers.column = "Sample_Name", import.idat.platform = "probesEPICv2") Errore in rnb.options(filtering.sex.chromosomes.removal = FALSE, identifiers.column = "Sample_Name", : invalid value for option import.idat.platform

despite following all the suggested steps.

In the rnb.options, "probesEPICv2" is not listed in the "import.idat.platform" section.

bariskalem commented 5 months ago

Hi @LucianoCalzari,

That's unexpected! Can you try installing RnBeads using devtools' install_github function. See below:

library(devtools)
install_github("epigen/RnBeads@feature/EPICv2")
library(RnBeads)

The following line shouldn't throw an error: rnb.options(import.idat.platform = "probesEPICv2")

Let us know if the problem persists.

LucianoCalzari commented 5 months ago

Hi @bariskalem, Thank you for your kind response. I really appreciate it. I followed your suggestion for installing RnBeads (previously, I was using `install_github("epigen/RnBeads")). Forgive me, as I am a beginner. I also reinstalled hg38 on R version 4.3.2 RC. Before the run command I used rnb.get.annotation(type = "CpG", assembly = "hg38") However, I encountered a new error, and I'm sorry to bother you with this:

result <- rnb.run.import(data.source=data.source, data.type="idat.dir", dir.reports=report.dir) 2024-02-07 09:05:34 Inf STATUS STARTED Loading Data 2024-02-07 09:05:34 Inf INFO Number of cores: 1 2024-02-07 09:05:34 Inf INFO Loading data of type "idat.dir" 2024-02-07 09:05:34 Inf STATUS STARTED Loading Data from IDAT Files 2024-02-07 09:05:34 Inf INFO Added column barcode to the provided sample annotation table 2024-02-07 09:05:34 Inf INFO Detected platform: MethylationEPICv2 2024-02-07 09:05:34 Inf INFO Annotation package genome assembly version: hg38 2024-02-07 09:07:04 Inf STATUS COMPLETED Loading Data from IDAT Files Errore in order(...) : l'argomento 1 non è un vettore Error in order(...): argument 1 is not a vector

These new Epic V2 will be a nightmare for everyone for a while..

Thank you in advance for your valuable assistance

bariskalem commented 5 months ago

Hi @LucianoCalzari ,

It's really difficult to pinpoint what the issue might be without seeing your analysis script or the traceback() of the error, but I'll try my best.

First of all, rnb.get.annotation(type = "CpG", assembly = "hg38") just retrieves the locations of each CpG dinucleotide in the hg38 genome. If you are interested in EPIC v2 probe locations try EPICv2.anno <- rnb.get.annotation(type = "probesEPICv2", assembly = "hg38")

I don't see how you're using this function for your analysis. That might be causing the problem.

Again, if you can go ahead and type traceback() after the error and send us the output, that would help us to see what might be causing the problem.

LucianoCalzari commented 5 months ago

Hi @bariskalem, thanks again. My main objective is to import from IDAT files and verify the sex

This is my script: library(RnBeads) data.dir <- "C:/XXXX/" idat.dir <- file.path(data.dir, "idat") sample.annotation <- file.path(data.dir, "SS_XXXX_02022024.csv") analysis.dir <- "C:/XXXX/" report.dir <- file.path(analysis.dir, "Import_forSEX") rnb.initialize.reports(report.dir) rnb.options(filtering.sex.chromosomes.removal = FALSE, identifiers.column = "Sample_Name", import.idat.platform = "probesEPICv2") logger.start(fname = NA) data.source <- list(idat.dir, sample.annotation) result <- rnb.run.import(data.source=data.source, data.type="idat.dir", dir.reports=report.dir)

This is my traceback()

traceback() 10: order(...) 9: eval(mc, env) 8: eval(mc, env) 7: eval(mc, env) 6: standardGeneric("order") 5: order(probe_names[dup_ind]) 4: read.idat.files(base.dir = data.source[[1]], sample.sheet = data.source[[2]], useff = rnb.getOption("disk.dump.big.matrices"), dpval.method = rnb.getOption("import.dpval.method"), verbose = verbose) 3: rnb.execute.import(data.source, data.type) 2: rnb.step.import(data.source, data.type, report) 1: rnb.run.import(data.source = data.source, data.type = "idat.dir", dir.reports = report.dir)

Hoping I've done everything you asked... thanks and best regards Luciano

bariskalem commented 5 months ago

Hi @LucianoCalzari ,

I'm suspecting you did not install the annotation package correctly. Or using an older build.

Please download this: https://icbb-share.s3.eu-central-1.amazonaws.com/RnBeads/EPICv2/RnBeads_hg38_preview/RnBeads.hg38_1.35.0.tar.gz

And then install the package by this line: install.packages("RnBeads.hg38_1.35.0.tar.gz", repos = NULL, type="source")

For good measure, make sure to run library(RnBeads.hg38) in your analysis script.

Please let me know if that solves the problem.

LucianoCalzari commented 5 months ago

Hi @bariskalem, Thank you for the suggestion. In fact, in my haste to uninstall and reinstall everything, I didn't reinstall version 1.35 (which I had previously used) but version 1.34. Now it works. I can only apologize and say a big, big thank you!

bariskalem commented 5 months ago

@LucianoCalzari no worries, glad we worked it all out!

marioelaprendiz commented 4 months ago

Do you know how to combine EPIC and EPICv2 samples in the same analysis? How can I create an rnb.set object that contains both types of samples? Does combine.rnb.sets combine EPIC and EPICv2 data.sets?

bariskalem commented 4 months ago

@marioelaprendiz

Do you know how to combine EPIC and EPICv2 samples in the same analysis? How can I create an rnb.set object that contains both types of samples? Does combine.rnb.sets combine EPIC and EPICv2 data.sets?

You can use the rnb.combine.arrays function to combine two RnBeadSet objects:

combined.arr = rnb.combine.arrays(rnb.set.epicv1, rnb.set.epicv2, type = "common")

marioelaprendiz commented 4 months ago

The function return me: Error in rnb.combine.arrays(x, y, type = type) : Incompatible assemblies is it because EPIC data uses hg19 and EPICv2 uses hg38?

> rnb.set = combine.rnb.sets(rnb.set.oldAC,rnb.set)
Error in rnb.combine.arrays(x, y, type = type) : Incompatible assemblies
> rnb.set.oldAC
Object of class RnBeadRawSet
     236 samples
  672438 probes
    of which: 672438 CpG, 0 CpH, 0 rs and 0 nv
Region types:
      221013 regions of type tiling
       31172 regions of type genes
       40596 regions of type promoters
       24348 regions of type cpgislands
Intensity information is present
Detection p-values are present
Bead counts are present
Quality control information is present
Summary of normalization procedures:
    The methylation data was normalized with method wm.dasen.
    No background correction was performed.
> rnb.set
Object of class RnBeadRawSet
     108 samples
  766828 probes
    of which: 766262 CpG, 0 CpH, 0 rs and 566 nv
Region types:
      238789 regions of type tiling
       35923 regions of type genes
       42721 regions of type promoters
       23999 regions of type cpgislands
Intensity information is present
Detection p-values are present
Bead counts are present
Quality control information is present
Summary of normalization procedures:
    The methylation data was normalized with method wm.dasen.
    No background correction was performed.
bariskalem commented 4 months ago

@marioelaprendiz

Yes. Set the following option before importing the data: rnb.options(assembly="hg38")

AgiaJorda commented 2 months ago

I just found this thread, and it is really helpful. I wanted to ask if RnBeads can still annotate to hg19 with EPICv2? I have done a large analysis for hg19, as many of the probes on the array were designed for that genome. The same can still be said for EPICv2 I believe, so if possible I would still like to annotate to hg19

AgiaJorda commented 2 months ago

Hi Guys, I am also running into an issue with the vanilla analysis during the exploration of the data. Specifically at the Sample Beta Value distributions - Sample Groups.

Here is the error I am getting: objc[50984]: +[NSConstantIntegerNumber initialize] may have been in progress in another thread when fork() was called. objc[50984]: +[NSConstantIntegerNumber initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug. objc[50985]: +[NSConstantIntegerNumber initialize] may have been in progress in another thread when fork() was called. objc[50985]: +[NSConstantIntegerNumber initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug. objc[50989]: +[NSConstantIntegerNumber initialize] may have been in progress in another thread when fork() was called. objc[50989]: +[NSConstantIntegerNumber initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug. objc[50991]: +[NSConstantIntegerNumber initialize] may have been in progress in another thread when fork() was called. objc[50991]: +[NSConstantIntegerNumber initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug. objc[50987]: +[NSConstantIntegerNumber initialize] may have been in progress in another thread when fork() was called. objc[50987]: +[NSConstantIntegerNumber initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug. Error in rnb.add.figure(report, description, figure.plots, setting.names) : invalid value for report.plots

I tried jrnb.options(exploratory.beta.distribution = FALSE) but it doesnt work in the new nv probe section for EPIC v2. Any advise on how to get around this?

AgiaJorda commented 2 months ago

So just to update, I found out how to turn this off, but I still get this error if I have multiple differential comparison columns, so i need to do one comparison at a time to get the pipeline to work with EPICv2