`pheweb process --no-parse` is killed during the processing

[Shicheng-Guo]

Hi Peter,

pheweb process --no-parse is killed during the processing to Manhattan plot. Problem is 18 jobs are processing at same time. How to figure out which file has problem? I checked the file /projects/ps-janssen4/dsci-csb/user/sguo2/pheweb/generated-by-pheweb/tmp/exception-2020-11-03T14-45-44.434745, nothing is useful to figure out how to solve the problem.

Thanks.

Shicheng

(base) [sguo2@comet-ln2 pheweb]$ pheweb process --no-parse
==> Starting `pheweb phenolist verify`
The 1306 phenotypes in '/projects/ps-janssen4/dsci-csb/user/sguo2/pheweb/pheno-list.json' look good.
==> Completed in 0 seconds

==> Starting `pheweb sites`
The list of sites is up-to-date!
==> Completed in 1 seconds

==> Starting `pheweb make-gene-aliases-trie`
gene aliases are at '/home/sguo2/.pheweb/cache/gene_aliases-v29-hg19.marisa_trie'
==> Completed in 0 seconds

==> Starting `pheweb add-rsids`
rsid annotation is up-to-date!
==> Completed in 0 seconds

==> Starting `pheweb add-genes`
gene annotation is up-to-date!
==> Completed in 0 seconds

==> Starting `pheweb make-tries`
tries are up-to-date!
==> Completed in 0 seconds

==> Starting `pheweb augment-phenos`
Output files are all newer than input files, so there's nothing to do.
==> Completed in 1 seconds

==> Starting `pheweb manhattan`
Processing 538 phenos (768 already done)
Completed    0 tasks in 0 seconds (18 currently in progress, 538 remain)
==> failed after 122 seconds
Child process had exception, info dumped to /projects/ps-janssen4/dsci-csb/user/sguo2/pheweb/generated-by-pheweb/tmp/exception-2020-11-03T14-45-44.424885
(Details in /projects/ps-janssen4/dsci-csb/user/sguo2/pheweb/generated-by-pheweb/tmp/exception-2020-11-03T14-45-44.434745)

[pjvandehaar]

It sounds like you had a pvalue of 0 (or <10^-320), and python doesn't know how to take the log of that. I'll look into why pheweb isn't handling that.

[pjvandehaar]

Oh, it sounds like you had so many variants with pvalue < 10^-320 that pheweb had to bin some of them. That's quite surprising. I'm not sure what to do about that, but I'll work on a fix.

[pjvandehaar]

I modified the manhattan-binning step to allow binning of variants with pval=0 (nlpval=inf). I haven't looked at the effects on the d3 plotting javascript yet. But at least this issue is resolved, and if the javascript needs changes that should be simple later.