how to prevent parallel submission from removing LD_ environment variables

[sckinta]

Hi I am using HiC-Pro parallel on SGE cluster. Usually it requires loading modules at job submission. For example, in HiC- Pro specifically, I have to add "module load python/2.7" to HiCPro_step1.sh and HiCPro_step2.sh allow cluster to run python 2.7. However this was saved in environment variables LD_LIBRARY_PATH which was removed when qsub HiCPro_step1.sh.

removed environment variable LD_PRELOAD from submit environment - it is considered a security issue
removed environment variable LD_LIBRARY_PATH from submit environment - it is considered a security issue
removed environment variable LD_LIBRARY_PATH_modshare from submit environment - it is considered a security issue
Your job-array 18519783.1-26:1 ("HiCpro_s1_ArimaHiC") has been submitted

My understanding is make makefile command in HiCPro_step1.sh and HiCProstep2.sh by default forces removing environment variable LD. Since besides python/2.7, we have many more softwares (like samtools bowtie ...) depend on LD that were saved in LD_ variables. HiCPro crashed without those variables being set. eg.

Bowtie2 alignment step1 ...
Logs: logs/High_Arima_HiC_lib_3/mapping_step1.log
/mnt/isilon/sfgi/programs/miniconda3/bin/samtools: error while loading shared libraries: libbz2.so.1.0: cannot open shared object file: No such file or directory
/mnt/isilon/sfgi/programs/miniconda3/bin/samtools: error while loading shared libraries: libbz2.so.1.0: cannot open shared object file: No such file or directory

Is there any way I can get around it, preventing job submission from removing LD_* environment variables?

[nservant]

Hi, I'm afraid I cannot really help you as I have no experience at all on that question. If you need to do any patch, you can modify the script make_sge_script.sh which generate the qsub scripts.