KeyError: 'ARNIEFILE'

[maya123z]

I attempted to run the conservation.py script with no alterations. (I had to install arnie in the same directory to get it to run.) However it gives me this error:

Traceback (most recent call last): File "conservation.py", line 4, in from sarscov2_util import * File "/oasis/tscc/scratch/mgosztyl/covid19_cirts/SARSCoV2_Secstruct_Cons/sarscov2_util.py", line 2, in from arnie.pfunc import pfunc File "/oasis/tscc/scratch/mgosztyl/covid19_cirts/SARSCoV2_Secstruct_Cons/arnie/pfunc.py", line 10, in package_locs = load_package_locations() File "/oasis/tscc/scratch/mgosztyl/covid19_cirts/SARSCoV2_Secstruct_Cons/arnie/utils.py", line 459, in load_package_locations with open("%s" % os.environ["ARNIEFILE"],'r') as f: File "/home/mgosztyl/miniconda3/lib/python3.7/os.py", line 679, in getitem raise KeyError(key) from None KeyError: 'ARNIEFILE'

[everyday847]

I haven't installed this package yet, but a keyerror in os.environ is an environment variable that hasn't been set. The code should definitely be giving you a more informative error message! But let's see, looking at the arnie source I see where your error is coming from:

def load_package_locations():
    '''Read in user-supplied file to specify paths to RNA folding packages. Specify this in your ~/.bashrc as $ARNIEFILE'''

It looks like you need a file -- whose filename is saved to $ARNIEFILE in your shell environment -- that gives the paths to all the folding engines you might want to use.

In the installation instructions at https://github.com/DasLab/arnie, it looks like it suggests you can imitate https://github.com/DasLab/arnie/blob/master/example_arnie_file.txt.

I hope that helps!

[maya123z]

I created a file like this and added it to my .bashrc (and sourced .bashrc) but still got the same error.

On May 15, 2020, at 6:18 PM, Andy Watkins notifications@github.com wrote:

I haven't installed this package yet, but a keyerror in os.environ is an environment variable that hasn't been set. The code should definitely be giving you a more informative error message! But let's see, looking at the arnie source I see where your error is coming from:

def load_package_locations(): '''Read in user-supplied file to specify paths to RNA folding packages. Specify this in your ~/.bashrc as $ARNIEFILE''' It looks like you need a file -- whose filename is saved to $ARNIEFILE in your shell environment -- that gives the paths to all the folding engines you might want to use.

In the installation instructions at https://github.com/DasLab/arnie https://github.com/DasLab/arnie, it looks like it suggests you can imitate https://github.com/DasLab/arnie/blob/master/example_arnie_file.txt https://github.com/DasLab/arnie/blob/master/example_arnie_file.txt.

I hope that helps!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DasLab/SARSCoV2_Secstruct_Cons/issues/1#issuecomment-629567594, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI3YSIWA5DR76G5DAJR73J3RRXSXFANCNFSM4NCVKRLA.

[everyday847]

What's the output of env on the command line? The error you're seeing is specifically related to os.environ, the standard python module, not seeing a particular environment variable; if you have successfully set the $ARNIEFILE environment variable and os.environ isn't seeing it, then that would reflect a very concerning issue with your python environment writ large.

[maya123z]

This is the output of env. Not sure what I’m looking for here.

AS=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-as MKLROOT=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/mkl RSTUDIO_WHICH_R=/home/mgosztyl/miniconda3/bin/R LDFLAGS=-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/home/mgosztyl/miniconda3/lib -Wl,-rpath-link,/home/mgosztyl/miniconda3/lib -L/home/mgosztyl/miniconda3/lib AR=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-ar MANPATH=/opt/intel/2018.1.163/documentation_2018/en/man/common:/opt/torque/man: GCC_NM=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-gcc-nm HOSTNAME=tscc-1-50.sdsc.edu PBS_VERSION=TORQUE-4.2.10 IPPROOT=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/ipp INTEL_LICENSE_FILE=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/licenses:/opt/intel/licenses:/root/intel/licenses HOST=x86_64-conda_cos6-linux-gnu L_MKL_TEMP=M_VERS TERM=xterm-256color SHELL=/bin/bash NM=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-nm HISTSIZE=1000 CPPFLAGS=-DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/mgosztyl/miniconda3/include TMPDIR=/scratch/mgosztyl/21765930.tscc-mgr7.local GDBSERVER_MIC=/opt/intel/2018.1.163/debugger_YEAR/gdb/targets/mic/bin/gdbserver PBS_JOBNAME=STDIN CONDA_SHLVL=1 LIBRARY_PATH=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64_lin:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/ipp/../compiler/lib/intel64:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/ipp/lib/intel64:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/tbb/lib/intel64/gcc4.4 CONDA_PROMPT_MODIFIER=(base) GSETTINGS_SCHEMA_DIR_CONDA_BACKUP= SIZE=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-size FPATH=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/mkl/include PBS_ENVIRONMENT=PBS_INTERACTIVE MIC_LD_LIBRARY_PATH=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/compiler/lib/mic:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/mic:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/tbb/lib/mic ROCKSROOT=/opt/rocks/share/devel PBS_O_WORKDIR=/oasis/tscc/scratch/mgosztyl/covid19_cirts/SARSCoV2_Secstruct_Cons GFORTRAN=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-gfortran ANT_HOME=/opt/rocks PBS_TASKNUM=1 USER=mgosztyl LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:.tar=38;5;9:.tgz=38;5;9:.arc=38;5;9:.arj=38;5;9:.taz=38;5;9:.lha=38;5;9:.lz4=38;5;9:.lzh=38;5;9:.lzma=38;5;9:.tlz=38;5;9:.txz=38;5;9:.tzo=38;5;9:.t7z=38;5;9:.zip=38;5;9:.z=38;5;9:.Z=38;5;9:.dz=38;5;9:.gz=38;5;9:.lrz=38;5;9:.lz=38;5;9:.lzo=38;5;9:.xz=38;5;9:.bz2=38;5;9:.bz=38;5;9:.tbz=38;5;9:.tbz2=38;5;9:.tz=38;5;9:.deb=38;5;9:.rpm=38;5;9:.jar=38;5;9:.war=38;5;9:.ear=38;5;9:.sar=38;5;9:.rar=38;5;9:.alz=38;5;9:.ace=38;5;9:.zoo=38;5;9:.cpio=38;5;9:.7z=38;5;9:.rz=38;5;9:.cab=38;5;9:.jpg=38;5;13:.jpeg=38;5;13:.gif=38;5;13:.bmp=38;5;13:.pbm=38;5;13:.pgm=38;5;13:.ppm=38;5;13:.tga=38;5;13:.xbm=38;5;13:.xpm=38;5;13:.tif=38;5;13:.tiff=38;5;13:.png=38;5;13:.svg=38;5;13:.svgz=38;5;13:.mng=38;5;13:.pcx=38;5;13:.mov=38;5;13:.mpg=38;5;13:.mpeg=38;5;13:.m2v=38;5;13:.mkv=38;5;13:.webm=38;5;13:.ogm=38;5;13:.mp4=38;5;13:.m4v=38;5;13:.mp4v=38;5;13:.vob=38;5;13:.qt=38;5;13:.nuv=38;5;13:.wmv=38;5;13:.asf=38;5;13:.rm=38;5;13:.rmvb=38;5;13:.flc=38;5;13:.avi=38;5;13:.fli=38;5;13:.flv=38;5;13:.gl=38;5;13:.dl=38;5;13:.xcf=38;5;13:.xwd=38;5;13:.yuv=38;5;13:.cgm=38;5;13:.emf=38;5;13:.axv=38;5;13:.anx=38;5;13:.ogv=38;5;13:.ogx=38;5;13:.aac=38;5;45:.au=38;5;45:.flac=38;5;45:.mid=38;5;45:.midi=38;5;45:.mka=38;5;45:.mp3=38;5;45:.mpc=38;5;45:.ogg=38;5;45:.ra=38;5;45:.wav=38;5;45:.axa=38;5;45:.oga=38;5;45:.spx=38;5;45:*.xspf=38;5;45: LD_LIBRARY_PATH=/opt/openmpi/intel/ib/lib:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64_lin:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/ipp/../compiler/lib/intel64:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/ipp/lib/intel64:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/tbb/lib/intel64/gcc4.4:/opt/sdsc/lib CONDA_BACKUP_HOST=x86_64-conda_cos6-linux-gnu PBS_O_HOME=/home/mgosztyl CONDA_EXE=/home/mgosztyl/miniconda3/bin/conda MIC_LIBRARY_PATH=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/compiler/lib/mic:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/tbb/lib/mic ROCKS_ROOT=/opt/rocks CPATH=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/ipp/include:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/mkl/include:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/tbb/include DEBUG_FORTRANFLAGS=-fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mgosztyl/miniconda3/include -fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fcheck=all -fbacktrace -fimplicit-none -fvar-tracking-assignments -ffunction-sections -pipe PBS_WALLTIME=14400 CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mgosztyl/miniconda3/include PBS_GPUFILE=/var/spool/torque/aux//21765930.tscc-mgr7.localgpu PBS_MOMPORT=15003 LD_GOLD=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-ld.gold CONDA_BUILD_SYSROOT=/home/mgosztyl/miniconda3/x86_64-conda_cos6-linux-gnu/sysroot STRINGS=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-strings CPP=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-cpp _CE_CONDA= PBS_O_QUEUE=home-yeo CXXFILT=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-c++filt NLSPATH=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64lin/locale/%l%t/%N:/opt/intel/2018.1.163/debuggerYEAR/gdb/intel64/share/locale/%l%t/%N MAIL=/var/spool/mail/mgosztyl PBS_O_LOGNAME=mgosztyl PATH=/home/mgosztyl/miniconda3/bin:/home/mgosztyl/miniconda3/condabin:/opt/openmpi/intel/ib/bin:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/bin/intel64:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/sdsc/bin:/opt/sdsc/sbin:/opt/maui/bin:/opt/torque/bin:/opt/torque/sbin:/opt/rocks/bin:/opt/rocks/sbin:/home/mgosztyl/miniconda3/bin:/home/mgosztyl/.local/bin:/home/mgosztyl/bin GSETTINGS_SCHEMA_DIR=/home/mgosztyl/miniconda3/share/glib-2.0/schemas PBS_O_LANG=en_US.UTF-8 SDSCDEVEL=/opt/sdsc/devel PBS_JOBCOOKIE=A61620F25CD9101BB4F797C0DD74AC6D DEBUG_CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -ffunction-sections -pipe -isystem /home/mgosztyl/miniconda3/include LD=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-ld CONDA_PREFIX=/home/mgosztyl/miniconda3 TBBROOT=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/tbb F90=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-gfortran PWD=/home/mgosztyl STRIP=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-strip LMFILES=/opt/modulefiles/compilers/intel/2018.1.163:/opt/modulefiles/mpi/openmpi_ib/3.1.4:/projects/ps-yeolab4/modulefiles/yeolabmodules/0.0.2 JAVA_HOME=/usr/java/latest ELFEDIT=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-elfedit F95=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-f95 GCC_RANLIB=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-gcc-ranlib GDB_CROSS=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/bin/intel64/gdb-mic PBS_NODENUM=0 LANG=C MODULEPATH=/projects/ps-yeolab4/modulefiles/.yeolab:/projects/ps-yeolab4/modulefiles/.tsccdigest:/projects/ps-yeolab4/modulefiles:/usr/share/Modules/modulefiles:/etc/modulefiles:/opt/modulefiles/applications:/opt/modulefiles/compilers:/opt/modulefiles/mpi:/opt/modulefiles/compilers:/opt/modulefiles/applications:/opt/modulefiles/mpi: LOADEDMODULES=intel/2018.1.163:openmpi_ib/3.1.4:yeolabmodules/0.0.2 PBS_NUM_NODES=1 DEBUG_FFLAGS=-fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mgosztyl/miniconda3/include -fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fcheck=all -fbacktrace -fimplicit-none -fvar-tracking-assignments -ffunction-sections -pipe F77=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-gfortran PBS_O_SHELL=/bin/bash LM_LICENSE_FILE=/opt/pgi/license.pgi:/opt/idl/license/license.idl PBS_JOBID=21765930.tscc-mgr7.local CXX=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-c++ _CE_M= MPM_LAUNCHER=/opt/intel/2018.1.163/debugger_YEAR/mpm/intel64_igfx/bin/start_mpm.sh HISTCONTROL=ignoredups OBJCOPY=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-objcopy INTEL_PYTHONHOME=/opt/intel/2018.1.163/debugger_YEAR/gdb/intel64/share/gdb/python SHLVL=1 HOME=/home/mgosztyl FORTRANFLAGS=-fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mgosztyl/miniconda3/include DEBUG_CPPFLAGS=-D_DEBUG -D_FORTIFY_SOURCE=2 -Og -isystem /home/mgosztyl/miniconda3/include CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mgosztyl/miniconda3/include ROLLSROOT=/opt/rocks/share/devel/src/roll PBS_O_HOST=tscc-login2.sdsc.edu FC=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-gfortran _CONDA_PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_x86_64_conda_cos6_linux_gnu GCC=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-gcc MPIHOME=/opt/openmpi/intel/ib OMPI_MCA_mpi_cuda_support=0 SDSCHOME=/opt/sdsc PBS_VNODENUM=0 ADDR2LINE=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-addr2line CONDA_PYTHON_EXE=/home/mgosztyl/miniconda3/bin/python PYTHONPATH=/opt/sdsc/lib LOGNAME=mgosztyl PBS_QUEUE=home-yeo MODULESHOME=/usr/share/Modules CONDA_DEFAULT_ENV=base MKL_ROOT=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/mkl LESSOPEN=||/usr/bin/lesspipe.sh %s PBS_MICFILE=/var/spool/torque/aux//21765930.tscc-mgr7.localmic PBS_O_SUBMIT_FILTER=/var/spool/torque/check.mail PBS_O_MAIL=/var/spool/mail/mgosztyl DEBUG_CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -ffunction-sections -pipe -isystem /home/mgosztyl/miniconda3/include RANLIB=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-ranlib INFOPATH=/opt/intel/2018.1.163/documentation_YEAR/en/debugger/gdb-ia/info/:/opt/intel/2018.1.163/documentation_YEAR/en/debugger/gdb-mic/info CMAKE_PREFIX_PATH=/home/mgosztyl/miniconda3:/home/mgosztyl/miniconda3/x86_64-conda_cos6-linux-gnu/sysroot/usr CC=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-cc PBS_NP=1 READELF=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-readelf PBS_NUM_PPN=1 PBS_O_SERVER=tscc-mgr7.local INCLUDE=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/mkl/include GCC_AR=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-gcc-ar OBJDUMP=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-objdump INTELHOME=/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux GPROF=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-gprof PBS_NODEFILE=/var/spool/torque/aux//21765930.tscc-mgr7.local GXX=/home/mgosztyl/miniconda3/bin/x86_64-conda_cos6-linux-gnu-g++ FFLAGS=-fopenmp -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mgosztyl/miniconda3/include PBS_O_PATH=/home/mgosztyl/miniconda3/bin:/home/mgosztyl/miniconda3/condabin:/opt/openmpi/intel/ib/bin:/opt/intel/2018.1.163/compilers_and_libraries_2018.1.163/linux/bin/intel64:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/sdsc/bin:/opt/sdsc/sbin:/opt/gold/bin:/opt/gold/sbin:.:/opt/maui/bin:/opt/torque/bin:/opt/torque/sbin:/opt/pdsh/bin:/opt/rocks/bin:/opt/rocks/sbin:/home/mgosztyl/miniconda3/bin:/home/mgosztyl/.local/bin:/home/mgosztyl/bin BASH_FUNCmodule()=() { eval /usr/bin/modulecmd bash $* } =/bin/env

On May 16, 2020, at 4:26 PM, Andy Watkins notifications@github.com wrote:

What's the output of env on the command line? The error you're seeing is specifically related to os.environ, the standard python module, not seeing a particular environment variable; if you have successfully set the $ARNIEFILE environment variable and os.environ isn't seeing it, then that would reflect a very concerning issue with your python environment writ large.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DasLab/SARSCoV2_Secstruct_Cons/issues/1#issuecomment-629719004, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI3YSIVLVO76ZAAY7BLT5FLRR4OL7ANCNFSM4NCVKRLA.

[everyday847]

Well, if you set the environment variable $ARNIEFILE in your .bashrc and source it, then one line in your env output should be ARNIEFILE=[something]. Your env output suggests that the issue is that you have not set the environment variable. What command did you use to try to set $ARNIEFILE?

[maya123z]

I inserted “ARNIEFILE=“/path/to/my/arnie_file.txt” in my .bashrc.

On May 16, 2020, at 4:57 PM, Andy Watkins notifications@github.com wrote:

Well, if you set the environment variable $ARNIEFILE in your .bashrc and source it, then one line in your env output should be ARNIEFILE=[something]. Your env output suggests that the issue is that you have not set the environment variable. What command did you use to try to set $ARNIEFILE?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DasLab/SARSCoV2_Secstruct_Cons/issues/1#issuecomment-629721426, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI3YSITQPCYC7SBC66E5XK3RR4R6DANCNFSM4NCVKRLA.

[everyday847]

In most bash-related shells, you need to use export ARNIEFILE=[whatever] in order for the environment variable assignment to persist.

The syntax you're using is intended for a sort of "local" context, discarded once the presently running script is over. (For example, it's common, when compiling programs with a specific compiler version that you don't want to use generally, to type something like CC=gcc-8.3 make -- once that command finishes, $CC will be whatever it had been beforehand.

Does that make sense?

[maya123z]

Thank you for your help, this solved that problem! However now I am getting a different error:

FileNotFoundError: [Errno 2] No such file or directory: ‘/path/arnie_software/contrafold/src/contrafold'

This is odd because the path in my arnie file is "/path/arnie_software/contrafold/src”, so I guess arnie is adding the extra “/contrafold” to the end. I tried changing the arnie file so now it just says “/path/arnie_software” and I get another error:

PermissionError: [Errno 13] Permission denied: ‘/path/arnie_software/contrafold'

On May 16, 2020, at 5:07 PM, Andy Watkins notifications@github.com wrote:

In most bash-related shells, you need to use export ARNIEFILE=[whatever] in order for the environment variable assignment to persist.

The syntax you're using is intended for a sort of "local" context, discarded once the presently running script is over. (For example, it's common, when compiling programs with a specific compiler version that you don't want to use generally, to type something like CC=gcc-8.3 make -- once that command finishes, $CC will be whatever it had been beforehand.

Does that make sense?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DasLab/SARSCoV2_Secstruct_Cons/issues/1#issuecomment-629722332, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI3YSIWIUWHWCZAUEJKWIDTRR4TFRANCNFSM4NCVKRLA.

[ramyarangan]

Hi Maya - The first approach you tried, adding ‘/your/path/to/contrafold/src' in the arnie file is correct, assuming that '/your/path/to/contrafold/' is changed to the directory where you installed Contrafold. If you installed Contrafold, you should have an executable called "contrafold" in that src directory. Can you run contrafold from that directory?

[maya123z]

Ah I see, it seems there is an error in my contrafold as it will not allow me to compile. When I run “make” it does not compile correctly. Have you experienced this issue before?

perl MakeDefaults.pl contrafold.params.complementary contrafold.params.noncomplementary contrafold.params.profile g++ -O3 -DNDEBUG -W -pipe -Wundef -Winline --param large-function-growth=100000 -Wall -c Contrafold.cpp In file included from LBFGS.hpp:52:0, from InnerOptimizationWrapper.hpp:12, from OptimizationWrapper.hpp:12, from Contrafold.cpp:16: LBFGS.ipp: In instantiation of 'Real LBFGS::Minimize(std::vector<_Tp>&) [with Real = double]': OptimizationWrapper.ipp:260:9: required from 'void OptimizationWrapper::LearnHyperparameters(std::vector, std::vector<_Tp>&) [with RealT = double]' Contrafold.cpp:451:9: required from 'void RunTrainingMode(const Options&, const std::vector&) [with RealT = double]' Contrafold.cpp:68:54: required from here LBFGS.ipp:112:105: error: 'DoLineSearch' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive] Real(0), std::min(Real(10), MAX_STEP_NORM / std::max(Real(1), Norm(d)))); ^ LBFGS.ipp:112:105: note: declarations in dependent base 'LineSearch' are not found by unqualified lookup LBFGS.ipp:112:105: note: use 'this->DoLineSearch' instead make: *** [Makefile:47: Contrafold.o] Error 1

On May 16, 2020, at 6:07 PM, Ramya Rangan notifications@github.com wrote:

Hi Maya - The first approach you tried, adding ‘/your/path/to/contrafold/src' in the arnie file is correct, assuming that '/your/path/to/contrafold/' is changed to the directory where you installed Contrafold. If you installed Contrafold, you should have an executable called "contrafold" in that src directory. Can you run contrafold from that directory?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DasLab/SARSCoV2_Secstruct_Cons/issues/1#issuecomment-629727404, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI3YSIQYTAWR2P3O2KDSDSDRR42ELANCNFSM4NCVKRLA.

[ramyarangan]

I unfortunately haven't encountered that error when installing Contrafold 2.0.

Depending on your goals, if the Contrafold 2.0 installation is proving tricky, one option might be to install a different secondary structure package (you can see the list of supported packages in the description at https://github.com/DasLab/arnie), add that package's path to the Arnie file, and change the string "contrafold_2" in sarscov2_util.py to refer to the new secondary structure package. In this script, Contrafold 2.0 is used to get a base-pair probability matrix for each region, but Arnie allows you to use other package as well to get that matrix.

[everyday847]

Maya, I've seen that error before. If you change line 112 of LBFGS.ipp to read this->DoLineSearch instead of DoLineSearch (your compiler is actually giving you this hint) then your contrafold will compile.

Contrafold is another lab's software, and it's not uncommon for software like this to be tested with only a subset of all possible compilers, which may have slightly different rules for this sort of function.

[maya123z]

Thanks so much for your help, I was able to get Contrafold working using te solution described here:

https://github.com/zavolanlab/MIRZAG/blob/master/docs/installation.rst

Final question (let me know if you'd prefer that I open a separate issue). I am trying to use this tool to identify longer target regions (40nt). Are these the only changes I would have to make to conservation.py in order for make that happen? (Stars around the changed sections; I tried to set the conserved intervals to 95%)

#line 109:
if np.sum(np.array(mean_cons) > **0.95**) > np.sum(np.array(rnd_mean_cons) > **0.95**):

#line 123:
ref_intervals = get_intervals_refseq(bcov_aln_counts, full_ref_seq, sequences, cutoff=1, **MIN_SIZE=35**)

#line 126:
all_intervals = get_all_secstructs_mea(ref_intervals, ref_seq, **secstruct_interval=40**)```

[ramyarangan]

Hi Maya,

This script works with two different types of sequence alignments: 1. Alignments of SARS-related coronaviruses that only includes the reference genome for the current outbreak, but includes closely related betacoronavirus genomes. 2. Alignments of SARS-CoV-2 sequences from the current outbreak.

To get intervals that are conserved in SARS-related sequences to at least 95% of size 40 nt, you can change line 123 to read ref_intervals = get_intervals_refseq(bcov_aln_counts, full_ref_seq, sequences, cutoff=0.95, MIN_SIZE=40). You can then look at the file that starts like results/conserved_intervals to get the sequences for the conserved intervals and the file that looks like results/intervals_full_...bed to get the interval ranges. You might need to go to lower than 95% conservation to get enough intervals of size 40.

To get intervals that are conserved in SARS-CoV-2 sequences and SARS-related sequences, you can change line 81 to read intervals = get_ref_intervals_from_file(sarsr_aln_file, cutoff=0.95, MIN_SIZE=40). The cutoff used in this line controls the level of sequence conservation expected in the SARS-related sequences. You then can also alter the conservation you'd like to enforce in the SARS-CoV-2 sequences by changing the percentage in line 153 (for instance, to if cur_min_cons > 0.95). Then, your updated intervals will be written to results/sars_related_conserved.csv. By the way, if you have a more recent sequence alignment for SARS-CoV-2 sequences from the current outbreak, you can update the sequence alignments in the alignments/ folder and change the file-names in this script accordingly.

For line 126, that controls the size of the window around the conserved region for which maximum expected accuracy structures are predicted. I would recommend keeping that to 20, so that secondary structures are predicted in a window with 20 nt on either side of the conserved regions.

Let me know if you have any more questions. Thanks so much for asking about this - I see that it would be helpful to change these scripts so these parameters are easier and more intuitive to manipulate.

[maya123z]

This is very helpful, thanks so much! Having these parameters as an input to the function would be a great addition. So if I wanted to do a similar thing for the unstructured.py script (i.e., find conserved unstructured regions >40 nt), would these be all the lines to change: 35, 65-67, 80-82, 106, 158, 160, 161, 166, 168. I tried to count up all the lines that include "min_size".

[ramyarangan]

Hi Maya - I've pushed an update that lets you modify these parameters more easily. You can now run a command like the following to get intervals of minimum length 40 where all bases are at most 50% likely to be paired, looking for intervals that are at least 90% conserved in SARS-CoV-2 sequences. You can change the thresholds as desired to get enough intervals.

python unstructured.py --min_len 40 --sarscov2_conservation 0.9 --min_unpaired_prob 0.5

For the conservation.py script, you can run some variant of the following to change interval lengths and desired conservation thresholds:

python conservation.py --min_len 40 --sarsr_conservation 0.8 --sarscov2_conservation 0.95

Running either script with -h will show the available options.

[maya123z]

This is a great update! Thanks for your help! Works great.

[maya123z]

Small question, what is contained in each column of the various output files? A header would be helpful to include. For example in sarscov2_conserved_unstructured.csv I'm not sure what columns 3 and 4 indicate.

[ramyarangan]

Updated to include headers for those files!

[maya123z]

Did you push this update? I'm not seeing it for some reason.

[ramyarangan]

Ah it should be pushed!