Still estimating CAUSE posteriors after 4 hrs

[Xuemin-Wang]

Dear Jean,

CAUSE returned the results for the MR analysis from trait A to B using default settings within about 20 mins. There were 9 unique elements of params_obj$mix_grid$S1 and 9 of params_obj$mix_grid$S2. CAUSE posteriors were estimated using 2062 variants. However, when I tried to estimate the causal relationship from trait B to A, it's still estimating posteriors after more than 4 hours with 4 cores (see images below). The number of unique elements of params_obj$mix_grid$S1 and params_obj$mix_grid$S2 were both 10. There were 1888 variants used in CAUSE posterior estimation.

I also tried to run it in HPC. It used as many as 27 cores but still couldn't finish the posteriors estimation after about 4 hours. So I had to kill the job due to too much resource usage. Below is the ganglia graph, which may or may not be helpful for your diagnosis.

Do you have any idea of what might be causing the issue?

Many thanks, patrick

[Xuemin-Wang]

I had a look at https://github.com/jean997/cause/issues/9 and https://github.com/jean997/cause/issues/5, but they both didn't provide an solution to my issue.

many thanks, patrick

[jean997]

Can you tell me how many variants you are using to estimate the parameters?

[Xuemin-Wang]

It's still estimating posteriors on my laptop after one day.

[jean997]

Yeah that is really long. How many variants did you use to estimate the parameters?

[jean997]

Ah sorry maybe I misunderstood. The posterior fitting step is taking a long time? That is not usually the long one. Usually the parameter estimation takes the longest.

[jean997]

Are you able to share your data? You could send me the parameters as an RDS file and the data just for the set of variants you are using to estimate posteriors so it shouldn't be too big. I'm not sure why this is happening for you.

[Xuemin-Wang]

Hi Jean,

Thanks for your reply.

It's the posterior estimation step that has been taking so long. It will be great if you can help investigate this issue. Here are the data. To_JeanM.zip

Many thanks, patrick

[ZixiaoWang17]

I met this problem too, the log remained in: 'Estimating CAUSE posteriors using XX variants.' for almost 12 hours in linux, and this phenomenon happened in almost 100+ cases.

[jean997]

Hi Zixao, How many variants are you using?

[ZixiaoWang17]

37 for example

Jean Morrison @.***> 于2021年11月16日周二 上午3:21写道：

Hi Zixao, How many variants are you using?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jean997/cause/issues/13#issuecomment-969239257, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWOFBJ6O5ZW6WFOGMQBB4KTUMFMUDANCNFSM4WURETSA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[jean997]

Hmm that is really odd. Would you be willing to share your data so I can see? @Xuemin-Wang I am looking at your example now though its been a while. I am sorry I missed your last message.

[jean997]

Ok @Xuemin-Wang and @ZixiaoWang17 I am not sure if you are having the same issue but I have some progress on @Xuemin-Wang 's issue. In that data, it seems that the problem is that the scale of beta_hat_1 is much different than the scale of beta_hat_2. I haven't identified exactly how this is causing an issue yet but it can be resolved by scaling up the estimates and sds for the study with very small estimates. In the case of @Xuemin-Wang 's data, I multiplied beta_hat_1 and sd1 by 100. I also scaled up the S1 column of params$mix_grid and this resolved the issue.

You could also scale up the data at the beginning before estimating the parameters and not have to modify the parameters object.

I just pushed an update to the package with some additional print statements for tracking progress. @ZixiaoWang17 can you use this version and see what step it is stalling on. I think if it is the same issue, it should be the "setting range" step. You can also try the scaling trick if your studies have estimates on very different scales.

I will keep digging and see if I can find an internal fix.

[jean997]

Ok -- I believe I solved the scale problem. Try the most recent version 1.2.0.0331 and see if your issues are resolved. If that doesn't solve it, let me know what step it is stalled on. Jean

[ZixiaoWang17]

Hi @jean997 ，Thanks for your great work and quick reply! I will give it a try and get back to you soon.

Zixiao

[ZixiaoWang17]

Hi @jean997 , I checked hundreds of cases and the problem is well solved! Thanks for your great contribution!

[jean997]

🙌