Error in Actual commands used for SuSiE paper

[ZacZZZ]

Hi Stephenslab,

I was trying to reproduce your result in your susieR repo

There are two errors I encounter. First regards the .yml files in both dsc susie.dsc --target run_comparison -o susie_comparison --host susie_comparison.yml -c 60 and dsc susie.dsc --target hard_case -o hard_case --host hard_case.yml -c 60 It returns

INFO: Load command line DSC sequence: run_comparison ERROR: Invalid configuration n_cpu in section default. Available configuations are: queue, instances_per_job, nodes_per_job, instances_per_node, cpus_per_instance, mem_per_instance, time_per_instance, prepend_path.

Second error occurs after I remove the --host option running dsc susie.dsc --target run_comparison -o susie_comparison -c 60

It returns

... [index=1]: Failed to execute Rscript .sos/get_sumstats_1_00455898.R exitcode=1, workdir=/path/to/susieRepo/susie-paper/src, stderr=susie_comparison/get_sumstats/liter_data_1_summarize_ld_1_lm_less_2_get_sumstats_1.stderr [DSC]: 5 pending steps: DSC, d_fit_caviar, c_plot_dap, b_plot_susie, a_fit_finemap WARNING: Files in green in the error prompt above contains codes and error info to help debug. Scripts upstream of the error can be found in susie_comparison.scripts.html. INFO: Elapsed time 48.145 seconds. ...

Any advice?

[gaow]

First regards the .yml files in both ...

You are not supposed to use those files anyways because as documented on the page, those files are specifically configured for SLURM system on the University of Chicago cluster. You have to tailor it for your own cluster system. See details here if you are interested:

https://stephenslab.github.io/dsc-wiki/advanced_course/Remote_Computations.html

Also make sure you're you on the latest version of as the interface to cluster system has changed. Please use pip install dsc -U to make sure. I'm not going to update the yml files posted though because they will work with the docker image to reproduce what we did, and it will not work for your cluster anyways. You are better off coming up with your own configuration as instructed.

Second error occurs

As prompted at the end of the message, could you look into, say, susie_comparison/fit_dap/liter_data_1_summarize_ld_1_lm_less_5_fit_dap_1.stderr to see what's going on? My guess is you haven't installed dap-g program? do you have it available from command line via dap-g? My benchmark code did not check that executable in particular because again the docker image has it. Even though you cannot run docker you can still check out into our dockerfile here:

https://github.com/stephenslab/susie-paper/blob/master/Dockerfile

to see exactly what programs are required and which version we used.

[ZacZZZ]

Hi gaow,

I now have the error for the .stderr files

Error: mkl-service + Intel(R) MKL: MKL_THREADING_LAYER=INTEL is incompatible with libgomp.so.1 library. Try to import numpy first or set the threading layer accordingly. Set MKL_SERVICE_FORCE_INTEL to force it.

Could you tell me how to implement the second line to solve this? I did some search but haven't found a solution yet.

It's presented in all the toy, susie comparison and hard case [Edit: I deleted the redundant error messages] Besides, the Dockerfile is really helpful! (I think telling people without root to refer to it to set up the environment on your repo will be helpful to others).

Have a great break!

Best, Zac

[gaow]

@ZacZZZ this line in the docker file should fix it I suppose? In your case you'd have to put the line in .bashrc with export MKL_THREADING_LAYER=GNU. Thanks for your suggestion regarding mentioning the dockerfile -- I've updated the instructions on the DSC experiment page.

I'm not sure about the other errors you run into. As previously suggested, you need to check contents of stderr files and fix them based on what you see there. Enjoy the holiday!

[pcarbo]

@gaow If this is an issue specific to the stephenslab/susie-paper repository, can we move the issue there?

[gaow]

@pcarbo there are two issues here:

1. Users are expected to look into the stderr file as prompted to figure out what's going on;
2. There is a more general issue with using mkl + numpy sometimes this can happen. DSC does use numpy so there can be this issue if the setup on user's end is not proper.

Both issues are generic issues but I'm not sure how to address to them ... I have improved the prompt in current DSC release hopefully users will be able to read it and realize they should look into the stderr files.