search

rwdavies / QUILT

GNU General Public License v3.0
45 stars 10 forks source link

HLA reference construction example error - check_mclapply_OK(out) #6

Closed rzeitoun-twist closed 2 years ago

rzeitoun-twist commented 2 years ago

Hi,

I am following the example QUILT_hla_reference_panel_construction.Md for building the HLA reference. I am not able to build the reference. After about 30 minutes it gives a check_mclapply_OK error. Would you happen to know where I should look for that? I double checked that my compressed reference files are not corrupted.

2021-09-16T23:27:08.447250573Z [2021-09-16 23:27:08] Using nMaxDH = 202 2021-09-16T23:27:09.491242429Z [2021-09-16 23:27:09] Save converted reference haplotypes 2021-09-16T23:27:09.686839066Z [2021-09-16 23:27:09] Done converting reference haplotypes 2021-09-16T23:27:14.463486109Z [2021-09-16 23:27:14] Using nMaxDH = 65 2021-09-16T23:27:14.765971807Z [2021-09-16 23:27:14] Save converted reference haplotypes 2021-09-16T23:27:14.884444931Z [2021-09-16 23:27:14] Done converting reference haplotypes 2021-09-16T23:27:17.844478428Z [2021-09-16 23:27:17] Using nMaxDH = 213 2021-09-16T23:27:18.862178250Z [2021-09-16 23:27:18] Save converted reference haplotypes 2021-09-16T23:27:19.039634287Z [2021-09-16 23:27:19] Done converting reference haplotypes 2021-09-16T23:27:19.040679487Z [2021-09-16 23:27:19] 2021-09-16T23:27:19.040909549Z Error in check_mclapply_OK(out) : 2021-09-16T23:27:19.040921178Z An error occured during QUILT. The first such error is above 2021-09-16T23:27:19.040926180Z Calls: QUILT_HLA_prepare_reference -> phase_hla_haplotypes -> check_mclapply_OK 2021-09-16T23:27:19.040931130Z Execution halted

For my system, I am running this in a Docker container built on r-base, specifically r-base:4.1.1@sha256:8da10a720b26d6b6c2d32cb743f90cb9bf54f4b536e471c3da23342510380fb7.

rwdavies commented 2 years ago

Thanks, I just re-ran this and indeed this just crashed on my machine, I'll have a look and figure out what broke it (Ps sorry about the 30 minutes, given that this is intended to be a one-off type analysis, it's not been optimized for speed)

rwdavies commented 2 years ago

OK, I think this should fix it https://github.com/rwdavies/QUILT/commit/b4b24bde6dc9897e8180e650ce3fe693bfe64828 IIRC, in the way that R easily does multiple core computing, if a sub-process fails due to e.g. running out of RAM, the job will return a NULL value (not a proper error as one would hope). I thought I always ran all the multiple core sections of the code returning non-NULL results but this seems to be a case where the intended behaviour was non-NULL but it wasn't captured because I don't have tests around this (and don't run it as a matter of course). I'm going to run it through the whole thing now and make sure this indeed fixes it. You can pull this change and rebuild ./scripts/build-and-install.R or wait for a new version with this change (I'll do this ~Monday)

rzeitoun-twist commented 2 years ago

@rwdavies thank you. That makes sense. It is no rush at all, was just trying out the whole QUILT experience for some of those genes other than the 5 included in the reference.