search

ChenMengjie / VIPER

MIT License
18 stars 4 forks source link

Error in imputation_by_samples_posterior_expectation #1

Open jhu99 opened 5 years ago

jhu99 commented 5 years ago

Hello Mengjie,

I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step.

The code is written as below:

system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt))

zerorateingene.pdf zerorateincell.pdf

Could you give me a guide to setup the parameter?

best, jialu

ChenMengjie commented 5 years ago

Hi Jialu,

It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells?

Thanks, Mengjie

Sent from my iPhone

On Feb 26, 2019, at 7:38 AM, screamer notifications@github.com<mailto:notifications@github.com> wrote:

Hello Mengjie,

I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step.

The code is written as below:

system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt))

zerorateingene.pdfhttps://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf zerorateincell.pdfhttps://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf

Could you give me a guide to setup the parameter?

best, jialu

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd.

jhu99 commented 5 years ago

Thanks Mengjie!

You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells?

Thanks, Mengjie

Sent from my iPhone

On Feb 26, 2019, at 7:38 AM, screamer notifications@github.com<mailto:notifications@github.com> wrote:

Hello Mengjie,

I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step.

The code is written as below:

system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt))

zerorateingene.pdfhttps://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf zerorateincell.pdfhttps://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf

Could you give me a guide to setup the parameter?

best, jialu

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

ChenMengjie commented 5 years ago

Hi Jialu,

That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset?

Thanks, Mengjie

On Feb 27, 2019, at 2:22 PM, screamer notifications@github.com<mailto:notifications@github.com> wrote:

Thanks Mengjie!

You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells?

Thanks, Mengjie

Sent from my iPhone

On Feb 26, 2019, at 7:38 AM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Hello Mengjie,

I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step.

The code is written as below:

system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt))

zerorateingene.pdfhttps://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf zerorateincell.pdfhttps://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf

Could you give me a guide to setup the parameter?

best, jialu

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-468035903, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd.

jhu99 commented 5 years ago

Hi Mengjie，

Thanks. I guess every gene has at least one zero. I will check it today and let you know. It’s 10x data for 293t cell line from Zheng’s publication.

Best, Jialu

发自我的 iPhone

在 2019年3月1日，上午12:03，ChenMengjie notifications@github.com 写道：

Hi Jialu,

That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset?

Thanks, Mengjie

On Feb 27, 2019, at 2:22 PM, screamer notifications@github.com<mailto:notifications@github.com> wrote:

Thanks Mengjie!

You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells?

Thanks, Mengjie

Sent from my iPhone

On Feb 26, 2019, at 7:38 AM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Hello Mengjie,

I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step.

The code is written as below:

system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt))

zerorateingene.pdfhttps://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf zerorateincell.pdfhttps://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf

Could you give me a guide to setup the parameter?

best, jialu

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-468035903, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

jhu99 commented 5 years ago

Hi Mengjie,

Every gene has at least one zero across all cells. More than 10k genes are not expressed in all 2850 cells. And each cell has at least one zero across all genes. And the sum of umi count for each cell is >0. I ran viper on both of the count matrix and the transpose of the count-matrix.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Friday, March 1, 2019 12:03 AM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset?

Thanks, Mengjie

On Feb 27, 2019, at 2:22 PM, screamer notifications@github.com<mailto:notifications@github.com> wrote:

Thanks Mengjie!

You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells?

Thanks, Mengjie

Sent from my iPhone

On Feb 26, 2019, at 7:38 AM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Hello Mengjie,

I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step.

The code is written as below:

system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt))

zerorateingene.pdfhttps://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf zerorateincell.pdfhttps://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf

Could you give me a guide to setup the parameter?

best, jialu

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-468035903, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

ChenMengjie commented 5 years ago

Hi Jialu,

Sorry I missed this message due to travels. Have you solved the problem? Let me summarize what we know so far and correct me if I’m wrong. The run on transposed matrix failed meaning using gene-to-gene relationship for prediction is failed. The response variable here is gene. The genes with all zeros will have constant values so that the regression can’t proceed. I would suggest to put this subset of genes aside and use others for the imputation. Because they are all zeros anyways.

Best, Mengjie

On Mar 1, 2019, at 8:58 AM, screamer notifications@github.com<mailto:notifications@github.com> wrote:

Hi Mengjie,

Every gene has at least one zero across all cells. More than 10k genes are not expressed in all 2850 cells. And each cell has at least one zero across all genes. And the sum of umi count for each cell is >0. I ran viper on both of the count matrix and the transpose of the count-matrix.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Friday, March 1, 2019 12:03 AM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset?

Thanks, Mengjie

On Feb 27, 2019, at 2:22 PM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Thanks Mengjie!

You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells?

Thanks, Mengjie

Sent from my iPhone

On Feb 26, 2019, at 7:38 AM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.commailto:notifications@github.com> wrote:

Hello Mengjie,

I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step.

The code is written as below:

system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt))

zerorateingene.pdfhttps://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf zerorateincell.pdfhttps://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf

Could you give me a guide to setup the parameter?

best, jialu

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-468035903, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-468692495, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417RojM3WAQ488PBRqxsTKJHfZFm65ks5vSUAQgaJpZM4bSUxd.

jhu99 commented 5 years ago

Thank you, Mengjie. It works well after filtering away all zero genes. Thanks again for your suggestion.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Thursday, March 14, 2019 1:14 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

Sorry I missed this message due to travels. Have you solved the problem? Let me summarize what we know so far and correct me if I’m wrong. The run on transposed matrix failed meaning using gene-to-gene relationship for prediction is failed. The response variable here is gene. The genes with all zeros will have constant values so that the regression can’t proceed. I would suggest to put this subset of genes aside and use others for the imputation. Because they are all zeros anyways.

Best, Mengjie

On Mar 1, 2019, at 8:58 AM, screamer notifications@github.com<mailto:notifications@github.com> wrote:

Hi Mengjie,

Every gene has at least one zero across all cells. More than 10k genes are not expressed in all 2850 cells. And each cell has at least one zero across all genes. And the sum of umi count for each cell is >0. I ran viper on both of the count matrix and the transpose of the count-matrix.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Friday, March 1, 2019 12:03 AM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset?

Thanks, Mengjie

On Feb 27, 2019, at 2:22 PM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Thanks Mengjie!

You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells?

Thanks, Mengjie

Sent from my iPhone

On Feb 26, 2019, at 7:38 AM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.commailto:notifications@github.com> wrote:

Hello Mengjie,

I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step.

The code is written as below:

system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt))

zerorateingene.pdfhttps://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf zerorateincell.pdfhttps://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf

Could you give me a guide to setup the parameter?

best, jialu

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-468035903, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-468692495, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417RojM3WAQ488PBRqxsTKJHfZFm65ks5vSUAQgaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

ChenMengjie commented 5 years ago

No problem. Please let me know if you have comments or suggestions on the method. We hope to improve VIPER so that it can be used in more settings. -Mengjie

On Mar 14, 2019, at 3:48 PM, screamer notifications@github.com<mailto:notifications@github.com> wrote:

Thank you, Mengjie. It works well after filtering away all zero genes. Thanks again for your suggestion.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Thursday, March 14, 2019 1:14 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

Sorry I missed this message due to travels. Have you solved the problem? Let me summarize what we know so far and correct me if I’m wrong. The run on transposed matrix failed meaning using gene-to-gene relationship for prediction is failed. The response variable here is gene. The genes with all zeros will have constant values so that the regression can’t proceed. I would suggest to put this subset of genes aside and use others for the imputation. Because they are all zeros anyways.

Best, Mengjie

On Mar 1, 2019, at 8:58 AM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Hi Mengjie,

Every gene has at least one zero across all cells. More than 10k genes are not expressed in all 2850 cells. And each cell has at least one zero across all genes. And the sum of umi count for each cell is >0. I ran viper on both of the count matrix and the transpose of the count-matrix.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Friday, March 1, 2019 12:03 AM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset?

Thanks, Mengjie

On Feb 27, 2019, at 2:22 PM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.commailto:notifications@github.com> wrote:

Thanks Mengjie!

You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells?

Thanks, Mengjie

Sent from my iPhone

On Feb 26, 2019, at 7:38 AM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.commailto:notifications@github.commailto:notifications@github.com> wrote:

Hello Mengjie,

I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step.

The code is written as below:

system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt))

zerorateingene.pdfhttps://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf zerorateincell.pdfhttps://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf

Could you give me a guide to setup the parameter?

best, jialu

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-468035903, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-468692495, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417RojM3WAQ488PBRqxsTKJHfZFm65ks5vSUAQgaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-473055630, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417VBTDXuTY4ikVsrBY1rq3ARyIk59ks5vWrWDgaJpZM4bSUxd.

jhu99 commented 5 years ago

I am happy to make some contribution to improve VIPER if possible. Recently, my work focuses on data alignment for single cell RNA-seq (classification for 10x data. I am open to discuss on any of these two problems.

Best Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Thursday, March 14, 2019 4:52 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

No problem. Please let me know if you have comments or suggestions on the method. We hope to improve VIPER so that it can be used in more settings. -Mengjie

On Mar 14, 2019, at 3:48 PM, screamer notifications@github.com<mailto:notifications@github.com> wrote:

Thank you, Mengjie. It works well after filtering away all zero genes. Thanks again for your suggestion.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Thursday, March 14, 2019 1:14 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

Sorry I missed this message due to travels. Have you solved the problem? Let me summarize what we know so far and correct me if I’m wrong. The run on transposed matrix failed meaning using gene-to-gene relationship for prediction is failed. The response variable here is gene. The genes with all zeros will have constant values so that the regression can’t proceed. I would suggest to put this subset of genes aside and use others for the imputation. Because they are all zeros anyways.

Best, Mengjie

On Mar 1, 2019, at 8:58 AM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Hi Mengjie,

Every gene has at least one zero across all cells. More than 10k genes are not expressed in all 2850 cells. And each cell has at least one zero across all genes. And the sum of umi count for each cell is >0. I ran viper on both of the count matrix and the transpose of the count-matrix.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Friday, March 1, 2019 12:03 AM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

That might not be because of zeros. You have a filter on zero percentage (if the cutoff is working). I guess some genes have the same value across all cells, not necessarily zero. Anyways, is this a smart-seq dataset?

Thanks, Mengjie

On Feb 27, 2019, at 2:22 PM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.commailto:notifications@github.com> wrote:

Thanks Mengjie!

You are right. Many genes in the input are zeros in all cells. However, I can’t filtered them away because the pretrained model were trained on a large number of cells. It needs all genes in the input matrix. Although I can filter them for viper, it will be difficult to track the masked value. Anyway, thanks for your reply.

Best, Jialu

Sent from Mail for Windows 10

From: ChenMengjie Sent: Wednesday, February 27, 2019 1:20 PM To: ChenMengjie/VIPER Cc: screamer; Author Subject: Re: [ChenMengjie/VIPER] Error inimputation_by_samples_posterior_expectation (#1)

Hi Jialu,

It looks like you have constant values in rows or columns. Can you do some filtering on genes so that you only keep genes that express in certain percentage of cells?

Thanks, Mengjie

Sent from my iPhone

On Feb 26, 2019, at 7:38 AM, screamer notifications@github.com<mailto:notifications@github.commailto:notifications@github.commailto:notifications@github.commailto:notifications@github.com> wrote:

Hello Mengjie,

I ran viper on a masked dataset (loaded in gene.expression, which is 28392 genes by 2850 cells). The zero rates hist plot of both genes and cells are in attached. I adjusted the default cutoff value according to the hist plot. I ran it on both gene.expression and its transpose. However, the first one ran smoothly and saved the output. The second one stopped unexpectedly. It prompts： Error in imputation_by_samples_posterior_expectation(data, selected_logxx, : Evaluation error: y is constant; gaussian glmnet fails at standardization step.

The code is written as below:

system.time(res <- VIPER(gene.expression, num = 5000, percentage.cutoff = 0.5, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefix)) system.time(res <- VIPER(t(gene.expression), num = 5000, percentage.cutoff = 0.88, minbool = FALSE, alpha = 0.5, report = TRUE, outdir = outdir, prefix = prefixt))

zerorateingene.pdfhttps://github.com/ChenMengjie/VIPER/files/2905929/zerorateingene.pdf zerorateincell.pdfhttps://github.com/ChenMengjie/VIPER/files/2905930/zerorateincell.pdf

Could you give me a guide to setup the parameter?

best, jialu

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417dwQFXuXA2TPvfazdLOZ1NHSd0p-ks5vRUb7gaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-468035903, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417XGTQ3kCTLAWm2rOJor_N5e6zaG3ks5vRvcQgaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-468692495, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417RojM3WAQ488PBRqxsTKJHfZFm65ks5vSUAQgaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/ChenMengjie/VIPER/issues/1#issuecomment-473055630, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AP417VBTDXuTY4ikVsrBY1rq3ARyIk59ks5vWrWDgaJpZM4bSUxd.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.