Error when running JDA dimensionality step

[danielgchen]

Hi, I was following your tutorial and everything seems to work great until the JDA dimension reduction step which came up with some errors shown below, I first tried the normalization method listed in the tutorial but since that wasn't an available method I switched to 'residual' because it was listed as one of the available normalization steps but that came up with the issue of JDA not having a method parameter, in another screen shot below? Do you know how I should change my code to run it? This is not 10x data, btw if that helps, thanks!

[r3fang]

Hi I am sorry for the trouble. The issue is because I started updating the package yesterday and it is still ongoing. Can you follow this tutorial:

https://github.com/r3fang/SnapATAC/blob/master/examples/10X_PBMC_15K/README.md https://github.com/r3fang/SnapATAC/blob/master/examples/10X_PBMC_15K/README.md

Let me know if it does not work

-- Rongxin Fang Ph.D. Student, Ren Lab Ludwig Institute for Cancer Research University of California, San Diego

On Aug 28, 2019, at 10:18 AM, danielgchen notifications@github.com wrote:

Hi, I was following your tutorial and everything seems to work great until the JDA dimension reduction step which came up with some errors shown below, I first tried the normalization method listed in the tutorial but since that wasn't an available method I switched to 'residual' because it was listed as one of the available normalization steps but that came up with the issue of JDA not having a method parameter, in another screen shot below? Do you know how I should change my code to run it? This is not 10x data, btw if that helps, thanks! https://user-images.githubusercontent.com/51771449/63877645-de0c8180-c97c-11e9-9f85-299ec8b3746d.png https://user-images.githubusercontent.com/51771449/63877846-3ba0ce00-c97d-11e9-9a97-781ae8e1da68.png — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/77?email_source=notifications&email_token=ABT6GG2YHHDFJ6ULBSGX7A3QG2XQHA5CNFSM4IRNIAOKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HH7QXDQ, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GGYJVX6CC7K5EACUAODQG2XQHANCNFSM4IRNIAOA.

[danielgchen]

Sounds good! Thank you!

[danielgchen]

In the tutorial it uses barcode.csvs however I currently don't have those, are they just like one column of barcode names? If so is the barcode just from creating the snap file, so in the screenshot below the beginning sequence at the start of each line:

[r3fang]

the barcode.csv file is just for barcode selection, you don’t need this for cell selection. Instead, you just can filter the cells based on the number of fragments etc.

-- Rongxin Fang Ph.D. Student, Ren Lab Ludwig Institute for Cancer Research University of California, San Diego

On Aug 28, 2019, at 11:49 AM, danielgchen notifications@github.com wrote:

In the tutorial it uses barcode.csvs however I currently don't have those, are they just like one column of barcode names? If so is the barcode just from creating the snap file, so in the screenshot below the beginning sequence at the start of each line: https://user-images.githubusercontent.com/51771449/63883856-f2a34680-c989-11e9-8e73-8307de1eeabd.png — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/77?email_source=notifications&email_token=ABT6GGYUQAK7QN3YOLQMOJ3QG3CFHA5CNFSM4IRNIAOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5MDMXI#issuecomment-525874781, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GGYT75U4HV4Y5LFPQPLQG3CFHANCNFSM4IRNIAOA.

[danielgchen]

Great. thanks!

[danielgchen]

If I skip the barcode step then I get an error in the subsequent step due to lack of information as shown below: I skipped these steps:

[r3fang]

that’s because you are still filtering the barcode using the single cell tsv file which is embedded in the snap object as metaData

-- Rongxin Fang Ph.D. Student, Ren Lab Ludwig Institute for Cancer Research University of California, San Diego

On Aug 28, 2019, at 12:22 PM, danielgchen notifications@github.com wrote:

in

[danielgchen]

Could you specify which lines of code is filtering the barcode in the code below?

library(SnapATAC)
library(GenomicRanges)

snap.files = "merged.snap"
sample.names = "day24"
x.sp = createSnap(
    file=snap.files,
    sample=sample.names
  )

x.sp@metaData["sample"] = x.sp@sample;
x.sp

row.covs.dens <- density(
  x = x.sp@metaData[,"logUMI"], 
  bw = 'nrd', adjust = 1
)
sampling_prob <- 1 / (approx(x = row.covs.dens$x, y = row.covs.dens$y, xout = x.sp@metaData[,"logUMI"])$y + .Machine$double.eps)
set.seed(1)
idx.landmark.ds <- sort(sample(x = seq(nrow(x.sp)), size = 10000, prob = sampling_prob))
x.landmark.sp = x.sp[idx.landmark.ds,]
x.query.sp = x.sp[-idx.landmark.ds,]

I thought this just loaded the Snap file and found landmarks: so just the creating snap poriton of step3 and step4:

[r3fang]

i think this step is wrong:

row.covs.dens <- density( x = x.sp@metaData[,"logUMI"], bw = 'nrd', adjust = 1 )

1) you are sampling based on the coverage but your x.sp@metaData object does not have a column called logUMI. Instead you can used log10(x.sp@metaData[“UQ"] +1); Does it work?

2) or you don’t need to sample landmarks if you have small number of cells. After filtering the low-quality cells, you just skip step 3-4 and treat x.sp as x.landmark.sp.

Let me know if you have more questions about it! Also, i will add another example where no landmark sampling is performed.

-- Rongxin Fang Ph.D. Student, Ren Lab Ludwig Institute for Cancer Research University of California, San Diego

On Aug 28, 2019, at 2:07 PM, danielgchen notifications@github.com wrote:

row.covs.dens <- density( x = x.sp@metaData[,"logUMI"], bw = 'nrd', adjust = 1 )

[danielgchen]

Thanks, I'll try that

[danielgchen]

Alright so when I change that to the code you gave it produces an error, if you could find another solution that would be great, but since I don't have much cells it's fine too.

[danielgchen]

Also when I run the diffusion map step I run into this error, this is after I set x.landmark.sp to x.sp more screenshots for reference

[danielgchen]

Also, when I plot the tsne graph and specify point.color to be cluster I don't get the clusters, as shown below the clusters parameter is filled, but nothing is shown in the graph

[r3fang]

Hi there,

1) the sampling issue should be fixed, please check the first and second example in the GitHub. https://github.com/r3fang/SnapATAC/blob/master/examples/10X_brain_5k/README.md https://github.com/r3fang/SnapATAC/blob/master/examples/10X_brain_5k/README.md https://github.com/r3fang/SnapATAC/blob/master/examples/10X_PBMC_15K/README.md https://github.com/r3fang/SnapATAC/blob/master/examples/10X_PBMC_15K/README.md

2) can you please check if x.sp@cluster has the cluster label?

Sorry that you just installed the package when I was updating it.

Best,

Rongxin Fang Ph.D. Student, Ren Lab Ludwig Institute for Cancer Research University of California, San Diego

On Aug 30, 2019, at 12:39 PM, danielgchen notifications@github.com wrote:

Also, when I plot the tsne graph and specify point.color to be cluster I don't get the clusters, as shown below the clusters parameter is filled, but nothing is shown in the graph https://user-images.githubusercontent.com/51771449/64047288-37aeb080-cb23-11e9-99bd-8d7f58d35fa1.png — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/77?email_source=notifications&email_token=ABT6GGZ53ZKWIC2WQBDL23LQHFZPXA5CNFSM4IRNIAOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5SSVDI#issuecomment-526723725, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GG73GMQIGNVATNLG443QHFZPXANCNFSM4IRNIAOA.

[danielgchen]

Hi,

No worries, so I followed the new 10x PBMC 15k tutorial and updated the package but I am still running into the same sampling issue for the DiffusionMap step as shown below: and in my session info it does say I updated SnapATAC

[r3fang]

Thanks! Can you reinstall the package and try it again?

-- Rongxin Fang, Ren Lab Ludwig Cancer Research Bioinformatics Ph.D. Student University of California, San Diego

On Sep 7, 2019, at 4:45 PM, danielgchen notifications@github.com wrote:

Hi,

No worries, so I followed the new 10x PBMC 15k tutorial and updated the package but I am still running into the same sampling issue for the DiffusionMap step as shown below: https://user-images.githubusercontent.com/51771449/64481321-d88a1500-d18e-11e9-82dc-2346dff2c849.png and in my session info it does say I updated SnapATAC https://user-images.githubusercontent.com/51771449/64481323-dfb12300-d18e-11e9-9113-edbedaa178ad.png — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/77?email_source=notifications&email_token=ABT6GG2SW7NCMQ4VKCWFJRTQIQ4KFA5CNFSM4IRNIAOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6FEMIQ#issuecomment-529155618, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GG7YJINELSEPJ2XJKB3QIQ4KFANCNFSM4IRNIAOA.

[danielgchen]

Also for the x.lda.sp@cluster this is what shows up: and when I look at the object here: there seems to be a cluster label:

[r3fang]

Again, the version you have downloaded was right in the middle of updates, if you can reinstall the package, I expect all these errors would be gone

On Sep 7, 2019, at 4:49 PM, danielgchen notifications@github.com wrote:

Also for the x.lda.sp@cluster this is what shows up: https://user-images.githubusercontent.com/51771449/64481329-1ab35680-d18f-11e9-93ca-1d81f5736351.png and when I look at the object here: https://user-images.githubusercontent.com/51771449/64481358-6960f080-d18f-11e9-9fd9-05ea5dedff04.png there seems to be a cluster label: https://user-images.githubusercontent.com/51771449/64481350-58b07a80-d18f-11e9-8b6c-349abff05e6f.png — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/77?email_source=notifications&email_token=ABT6GG6SMIRQ4NVE2KP336DQIQ4X5A5CNFSM4IRNIAOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6FENTQ#issuecomment-529155790, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GG5KQFHJ5BMLXDGLQDDQIQ4X5ANCNFSM4IRNIAOA.

[danielgchen]

Hi,

I tried to reinstall the package and got this: then I set the force parameter to true, but none of the previous issues were resolved

[r3fang]

can you repeat the first example? https://github.com/r3fang/SnapATAC/blob/master/examples/10X_brain_5k/README.md https://github.com/r3fang/SnapATAC/blob/master/examples/10X_brain_5k/README.md

On Sep 9, 2019, at 10:52 AM, danielgchen notifications@github.com wrote:

Hi,

I tried to reinstall the package and got this: https://user-images.githubusercontent.com/51771449/64554242-dbe8e200-d2ef-11e9-8b7b-7dc5193463d2.png then I set the force parameter to true, but none of the previous issues were resolved

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/77?email_source=notifications&email_token=ABT6GG2Q2A7I75WJEQR6XXTQI2EN3A5CNFSM4IRNIAOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6IPIPA#issuecomment-529593404, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GGZQ527ENPP24INNFV3QI2EN3ANCNFSM4IRNIAOA.

[danielgchen]

Hi, I just redownloaded the newest SnapATAC, from 13 hours ago, and diffusion map is working now and the clusters, thank you!

[danielgchen]

Hi,

When I try to run chromVAR it says the method is unavailable even though on Git it says it is

[danielgchen]

Hi,

I just downloaded your most recent update and now can run chromVAR, thank you! However, I am unable to recreate the plots that you had, when I try I get: using the code:

motif_i = "MA0909.1_HOXD13";
dat = data.frame(x=x.lda.sp@metaData[,"cluster"], y=x.lda.sp@mmat[,motif_i])
p1 <- ggplot(dat, aes(x=x, y=y, fill=x)) + 
  theme_classic() +
  geom_violin() + 
  xlab("cluster") +
  ylab("motif enrichment") + 
  ggtitle(motif_i) +
  theme(
    plot.margin = margin(5,1,5,1, "cm"),
    axis.text.x = element_text(angle = 90, hjust = 1),
    axis.ticks.x=element_blank(),
    legend.position = "none"
  )
p1

here is some information about the snap object: and the dat object looks like: any suggestions? thanks! aiming for plots like this, as shown in your guide:

[r3fang]

hi sorry for the late response!

this is a bit wired. Have you figured out this issue? can you remove theme and try it again

p1 <- ggplot(dat, aes(x=x, y=y, fill=x)) + theme_classic() + geom_violin() + xlab("cluster") + ylab("motif enrichment") + ggtitle(motif_i)

`

and let me know the outcome?

Rongxin Fang Ph.D. Student, Ren Lab Ludwig Institute for Cancer Research University of California, San Diego

On Sep 20, 2019, at 8:54 PM, danielgchen notifications@github.com wrote:

Hi,

I just downloaded your most recent update and now can run chromVAR, thank you! However, I am unable to recreate the plots that you had, when I try I get: https://user-images.githubusercontent.com/51771449/65365531-f9ab2680-dbce-11e9-8ebb-582fe29b8447.png using the code:

motif_i = "MA0909.1_HOXD13"; dat = data.frame(x=x.lda.sp@metaData[,"cluster"], y=x.lda.sp@mmat[,motif_i]) p1 <- ggplot(dat, aes(x=x, y=y, fill=x)) + theme_classic() + geom_violin() + xlab("cluster") + ylab("motif enrichment") + ggtitle(motif_i) + theme( plot.margin = margin(5,1,5,1, "cm"), axis.text.x = element_text(angle = 90, hjust = 1), axis.ticks.x=element_blank(), legend.position = "none" ) p1 here is some information about the snap object: https://user-images.githubusercontent.com/51771449/65365555-32e39680-dbcf-11e9-8d53-6070c4d6010a.png and the dat object looks like: https://user-images.githubusercontent.com/51771449/65365563-3f67ef00-dbcf-11e9-86a0-f5d4962071bd.png any suggestions? thanks! aiming for plots like this, as shown in your guide: https://user-images.githubusercontent.com/51771449/65365609-abe2ee00-dbcf-11e9-9a5c-b126aeef5f99.png — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/77?email_source=notifications&email_token=ABT6GGZTRGNIQ2N45ZM37DLQKVWEBA5CNFSM4IRNIAOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7IHBII#issuecomment-533754017, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GG5X7PMKTASHLOKQC6TQKVWEBANCNFSM4IRNIAOA.