`GLIBC_2.18' not found

[szhang0112]

I got this error when running the executable file: /glnexus_cli: /lib64/libc.so.6: version GLIBC_2.18 not found (required by ./glnexus_cli)

Since I ran it on cluster so the upgrade of glibc looks impossible. Is there other way to solve this problem? Docker was not installed in our cluster, so docker installation is also not available.

[mlin]

glibc 2.18 is so old that I'd doubt a newer version is truly absent from a reasonably managed cluster in 2020. May I suggest inquiring with the sysadmins if they're aware of any pitfalls with the environment setup which might cause programs to attempt to link a suboptimal libc. Otherwise I'm afraid we don't have bandwidth to ensure compatibility with 2013-era environments (but again I'm not yet convinced that's really the situation we're facing)

[szhang0112]

@mlin Thanks for the reply. Just solved the problem by using singularity instead. BTW, do you have any suggestions on running glnexus on very large cohort? Indeed I am running it for UK Biobank 50,000 exomes based on their initial processing.

[mlin]

Use --list to pass in a file containing the list of gVCF filenames, instead of trying to put 50K filenames on the command line.

And go through carefully: https://github.com/dnanexus-rnd/GLnexus/wiki/Performance

50K WES is a stretch for the open-source version, but it can be done (in several days) if you have a powerful compute node with dozens of threads and >100GB RAM, or if you can work out a scheme to split up all the gVCF files by chromosome to run separately. The DNAnexus-native version can distribute the processing across compute nodes but more efficiently without materializing the >1M little file slices implied there.

[szhang0112]

I got this error when using --list option [GLnexus] [error] Failed to read input file list: IOError: reading (*****/sample_list.txt)

[szhang0112]

@mlin More details about the error: when applying on two samples for a test, the error is still there: [E::hts_open_format] Failed to open file ukb/exomes/gvcf/3471475_23176_0_0.gz [47253] [2020-04-23 13:57:45.987] [GLnexus] [error] Failed to initialize database: IOError: Failed to open exemplar gVCF file at (ukb/exomes/gvcf/3471475_23176_0_0.gz)

[mlin]

I don't have too much direct experience with Singularity, but usually containers don't by default have access to arbitrary paths on the host file system, & you have to explicitly "bind mount" files/directories you want to use for input & output. I found this page: https://singularity.lbl.gov/docs-mount

[szhang0112]

@mlin I think it is working. Thank you so much for the help! This is what I did singularity run --bind [path] glnexus_v1.2.6.sif glnexus_cli [options] xxx for others' reference.

[szhang0112]

@mlin Is there any checkpointing feature for GLnexus? The job has been running for 15 days and it is almost there. However, the cluster is going to be maintained due to certain hardware issue and all jobs would be killed. I was wondering there is any way to avoid rerun the job? The current status is [309522] [2020-05-08 22:31:37.736] [GLnexus] [info] 41200/49953 (5543966_23176_0_0)...

[szhang0112]

@mlin I got this error unfortunately [309806] [2020-05-09 03:46:43.612] [GLnexus] [info] 41500/49953 (5198470_23176_0_0)... [308160] [2020-05-09 07:29:05.905] [GLnexus] [info] Loaded 41513 datasets with 41513 samples; 18625854271384 bytes in 180294920140 BCF records (108812551 duplicate) in 1556898955 buckets. Bucket max 213584 bytes, 1865 records. 0 BCF records skipped due to caller-specific exceptions [308160] [2020-05-09 15:46:24.930] [GLnexus] [error] Failed to bulk load into DB: IOError: RocksDB kIOError (IO error: While appending to file: GLnexus.DB/001569.sst: Input/output error)

[mlin]

@szhang0112 Sorry to hear that -- the last progress message you shared 41200/49953 (5543966_23176_0_0)... was actually only reporting the progress of the initial "bulk load" of the gVCF files into an internal database, so the subsequent processing had not even started yet.

While it's quite ambitious to try to push 50K exomes through a single compute job as I said, I do know of it having been done without taking weeks on end. What are the specs of the compute node, and were you able to go through all the items on Performance?

The DNAnexus-native version can do incremental/checkpointed processing as well as distributing across a large fleet of compute nodes, but these capabilities aren't provided in the open-source version. The best way to scale out the open-source version would be to make a custom pipeline to slice up all the gVCF files by chromosome (or even less) and run glnexus_cli separately on each slice.

[as7a5]

Hi Would somebody please list the commands/steps for installing Glnexus using singularity? Thanks

[szhang0112]

@mlin Any suggestions/tutorials to use GLnexus in DNANexus? We have DNANexus service and could move the job there. Thanks!

[leorippel]

@mlin I think it is working. Thank you so much for the help! This is what I did singularity run --bind [path] glnexus_v1.2.6.sif glnexus_cli [options] xxx for others' reference.

Hi @szhang0112

I'm having the same problem, but from how you wrote your command I could not understand how you used.

this is what I'm using

singularity run -B $PWD:/in glnexus_v1.2.2.sif glnexus_cli --config DeepVariant /in/M10.output.g.vcf.gz /in/M9.output.g.vcf.gz > $PWD/tst3.bcf

and i'm getting the same error

[36402] [2020-11-04 13:31:44.510] [GLnexus] [info] config CRC32C = 1435665943 [36402] [2020-11-04 13:31:44.510] [GLnexus] [info] init database, exemplar_vcf=/in/M10.output.g.vcf.gz [36402] [2020-11-04 13:31:44.510] [GLnexus] [error] Failed to initialize database: IOError: Database directory already exists (GLnexus.DB)

Can you give me any ideas how to solve this?

Cheers.

[leorippel]

Got it.

When using singularity, apparently, the scratch directory must not exist. So if you pass the flag --dir "somedir" it works.

Cheers.

[tuannguyen8390]

I don't have a luxury of running Singularity/Docker on my cluster. Did anyone have a solution for this error ?

Thanks,

[xiekunwhy]

@tuannguyen8390 no one solved this problem.

[ama249]

I just tried to install glnexus 1.4.1 via Bioconda, it installs libglib 2.76.4, glib-tools 2.76.4 & glib 2.76.4 as dependencies, but it still throws the error glnexus_cli: /lib64/libc.so.6: version GLIBC_2.18 not found (required by glnexus_cli).