Problem loading STITCH during installation

[boryanakis]

I am trying to install STITCH on a HPC cluster. I have a custom $R_LIBS directory for all my package installations. I have followed all the directions the repo's README about the installation and I run into the following error:

> R CMD INSTALL ./releases/STITCH_1.5.3.tar.gz

* installing to library ‘/panfs/pfs.local/work/sjmac/r_libraries’
* installing *source* package ‘STITCH’ ...
** libs

...

make[1]: Leaving directory `/tmp/Rtmp5GJ53X/R.INSTALL1ec264e985cb5/STITCH/src/SeqLib/htslib'
g++ -std=gnu++11 -shared -L/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/lib -L/home/b961k922/sjmac/software/stitch/STITCH/install/lib -o STITCH.so RcppExports.o bam_access.o bam_linker.o functions.o haploid.o heuristics.o hwe.o writers.o -L/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/lib -lRlapack -L/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/lib -lRblas -lgfortran -lm ./SeqLib//src/libseqlib.a ./SeqLib//htslib/libhts.a -fPIC -lz -L/panfs/pfs.local/software/install/MRO/3.5.1/ropen/3.5.1/lib64/R/lib -lR
installing to /panfs/pfs.local/work/sjmac/r_libraries/STITCH/libs
** R
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
Error: package or namespace load failed for ‘STITCH’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/panfs/pfs.local/work/sjmac/r_libraries/STITCH/libs/STITCH.so':
  /panfs/pfs.local/work/sjmac/r_libraries/STITCH/libs/STITCH.so: undefined symbol: lzma_easy_buffer_encode
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/panfs/pfs.local/work/sjmac/r_libraries/STITCH’

I have tried using Google to help me with this but I have hit a wall. Could you offer any suggestions?

[rwdavies]

Hi, can you post or link the full install log? This looks to be related to lzma, and by extension, SeqLib / hstlib. Can you try the steps around this part of the README configure: error: liblzma not found, please install lzma to install xz and see if that fixes it? Also, does hstlib install without issue or does it raise a similar complaint?

[boryanakis]

Hi. Thank you for getting back to me so quickly.

The log is attached. STITCH.installation.log

I have installed both lzma and xz in a conda environment. When that didn't work, I also installed xz in the manner that is described in the README.

I am happy to provide more information to help with troubleshooting.

[rwdavies]

Huh. This is tricky. Is this log from the de-activated conda environment? i.e. new terminal, just running cd /home/b961k922/sjmac/software/stitch/STITCH/ && R CMD INSTALL ./releases/STITCH_1.5.3.tar.gz? It's a longshot but just want to check it's not something weird between xz from conda vs what's in STITCH/install

[boryanakis]

No difference in the log (minus the /tmp/ directory names).

Are there specific gcc or R version requirements? I'm using R 3.5.1 and gcc 8.2.

[rwdavies]

There shouldn't be specific R or gcc requirements. 8.2 is impressively new. I'm still suspicious something has gone wrong with lzma / xz. I'm stumped about what though. Will try to come back to this tomorrow.

[rwdavies]

Sorry, couldn't get back to this yesterday. Upon further reading, the error message looks to relate to linking to the xz library (I had just assumed lzma_easy_buffer_encode was within htslib). Can you try once more on a clean install? Can you run the following (and correct any obvious bugs) to see if it runs on your system? (i.e. bash clean.txt (github won't let me upload .sh)) clean.txt

[boryanakis]

Thanks for looking at it again. I tried it anew (removed the old directory, started a new interactive job, purged all modules, loaded R and gcc, no conda env) with the script you attached. Unfortunately, it gives me the same error. I am attaching the log so that you can look at it.

stitch_clean_install.log

[rwdavies]

Hey, thanks for trying that previously. I must admit I'm stumped. Properly taking the time to debug this is probably equivalent to getting STITCH into bioconda or similar, so I'm looking at that now as a general solution I'm hoping will allow you to install STITCH and fix the majority of compilation problems. I'll keep you posted.

[rwdavies]

Hey, Happy New Year! I managed to get the depencency R library "rrbgen" into bioconda, which is good. Things moved a little slowly due to the holidays. I'll work on getting STITCH into bioconda now. I'm hoping that goes faster

[boryanakis]

Hi! Happy New Year to you as well! Thank you for working on this. r-rrbgen installs without any hiccups through bioconda. I am happy to test things out, if you need.

[rwdavies]

Hey, this took longer than I expected (see https://github.com/bioconda/bioconda-recipes/pull/12899), but STITCH should be on bioconda now. Can you try it out and let me know if it works for you?

[boryanakis]

Thanks for working on this! I installed the package through Conda without any errors. I loaded it as a library in R without issues, and finally ran the test described in the repo's README. The VCF was created as expected. A few notes about the process that could be useful for documenting the conda-related changes:

• the installation was flawless once I knew the package name (r-stitch)
• once I activated the environment, it wasn't clear to me how to run stitch from the command line (I eventually cloned the repo, and used the STITCH.R script)
• unless the bamlist.txt file is in the same dir as STITCH.R, the script fails (I think this is how the script behaves by default but I thought I'd mention it)

Overall, the Conda package seems to work great, and I regret not reaching out sooner.

Thanks a ton for doing this! I also have some feedback about the documentation. Would you like that in a separate issue? I could also wait until you release the next version and send you a PR.

[rwdavies]

Great! re: 2, yes, right, I didn't bundle in the "STITCH.R", as it currently installs just an R package. I'll explain this in the README. re: 3, if STITCH.R is in a different directory, it would need to be passed through to STITCH.R approapriately, e.g. "STITCH.R --bamlist= ...", with the rest of the example package in that same directory. Otherwise, it shouldn't need to be in the same directory?

Great that it works! I'm pleased I know how to do this and it indeed solves some of these horrible compilation problems. I spent forever once figuring out all the dependencies in scripts/install-package-dependencies.sh so I could install STITCH on a server in China without internet access running some bare-boned CentOS. Probably would have been much easier with conda.

Yes, if you don't mind, I'm going to close this issue, and you can open a new issue surrounding documentation. I added something about conda just now (e37544d608b6acc0896472999bb69672c0ae2820), but let me know if you have suggestions for improvements. A pull request or new issue would be great.

Thanks again for your patience with this