bash: module: command not found

[DilumAluthge]

Description

When I run module load conda/bcbi_v0.0.1 in a bash session inside Stronghold, I get the following error:

bash: module: command not found

Steps to Reproduce

1. Connect to Stronghold via FastX and open a new "ursa-ws gnome" session.

2. Once the desktop has loaded, click on Applications -> Terminal.

3. At the bash prompt, type module load conda/bcbi_v0.0.1 and press enter.

Expected Behaviour

The module loads.

Actual Behaviour

I receive the following error message:

bash: module: command not found

cc: @INSARKAR @eschen @mirestrepo @paulstey

[DilumAluthge]

Some additional information that might help:

1. The result of echo $PATH is:

   /usr/local/bin:/bin:/usr/bin

2. The result of which module is:

   which: no module in (/usr/local/bin:/bin:/usr/bin)

[DilumAluthge]

Alright, so at Isabel's suggestion I looked at the PyModules repo: https://bitbucket.org/mhowison/pymodules

The file PyModules/init/modules.sh (click here for source) reveals that module is actually a bash function. It also reveals that there should be an environment variable MODULEHOME that is set to /gpfs/runtime/pymodules.

But the plot thickens. There is no directory /gpfs/runtime/pymodules on Stronghold.

So I searched all of Stronghold for a directory named pymodules using the following command: find / -name "pymodules" 2> >(grep -v 'Permission denied' >&2)

And then I discovered the directory /opt/stronghold/pymodules. And inside there, there is a script file /opt/stronghold/pymodules/init/modules.sh

So I opened a new bash session and ran the following command:

source /opt/stronghold/pymodules/init/modules.sh

And I got the following output:

module: loading 'anaconda/2.3.0'
module: loading 'matlab/R2015a'
module: loading 'rstudio/0.98.1103'

So this is promising. Now I have a module command! So now I try module load conda/bcbi_v0.0.1. But this gives me the following error:

module: warning: unknown version 'conda/bcbi_v0.0.1'
  Please contact CIS support (cis-stronghold@brown.edu) if you think that
  software with this module name should be installed.

But unfortunately, I have no desire to contact CIS support. So instead, I run module avail, which gives me a lot of output. Under the category: environments section, I only see the following three entries:

conda/bcbi_v1
conda/root
conda/stronghold_v`

In particular, the environments conda/bcbi_v0.0.0 and conda/bcbi_v0.0.1 are not listed.

So instead, I run module load conda/bcbi_v1, which gives me the following output:

module: loading 'conda/bcbi_v1'

And now, I run which julia, and I receive the following output:

/opt/browncis/conda/envs/bcbi_v1/bin/julia

Now this is very interesting. It seems that all of the environments are stored under /opt/browncis/conda/envs.

So I do cd /opt/browncis/conda/envs and then I do ls and I get the following output:

ar_test
bcbi_v0.0.0
bcbi_v0.0.1
bcbi_v1
bcbi_v2
bjorkegren_v1
datasci_v1
datasci_v2
fgelintest
isabeltest
jucles
juclestest
psteytest
riipl_py3_v1
riipl_v1
riipl_v2
riipl_v3
riipl_v4
stronghold_v1

So here are all of the environments! So why does module load conda/bcbi_v0.0.1 not find the conda/bcbi_v0.0.1 environment, when a directory exists that is named /opt/browncis/conda/envs/bcbi_v0.0.1?

[DilumAluthge]

The saga continues:

So in bash, I run the following commands and receive the following output. I have put them in a table for your convenience.

Command	Output
echo $MODULEHOME	/opt/stronghold/pymodules
echo $MODULEPATH	/opt/stronghold/pymodules/modulefiles
echo $MODULESHELL	bash
echo $MODULEPYTHON	/opt/stronghold/anaconda/2.3.0/bin/python

Ok so it looks like something interesting lives in /opt/stronghold/pymodules/modulefiles.

So I run cd /opt/stronghold/pymodules/modulefiles. And then I run ls and I get the following output:

anaconda
atom
boost
conda
conda~
eclipse
emacs
encfs
gcc
gdal
gephi
git
haploview
iozone
java
julia
lyx
matlab
metal
mro
nodejs
parallel
plink
postgresql
probabel
R
rstudio
spatialindex
sqlcl
sqldeveloper
sqlite
stata
stata-ado
sublime_text
svn
unixodbc
unrar

So, out of that really long list, there are really only two that seem relevant to me: conda and conda~. So, let's take a look at each one of those.

So, first of all they are both files. If you run file conda you get the output conda: ASCII text, and if you run conda~ you get the output conda~: ASCII text. So they are plain text files, so we will just cat them.

First, let's look at conda. I run cat conda and receive the following output:

[DEFAULT]

name = conda
brief = Conda - package, dependency, and environment management.
url = https://conda.io/docs/
category = environments

rootdir = /opt/browncis/%(name)s/envs/%(version)s
prepend PATH = %(rootdir)s/bin

[root]
default = true
rootdir = /opt/browncis/%(name)s
prepend PATH = %(rootdir)s/bin

[stronghold_v1]

[bcbi_v1]

And let us also look at conda~. I run cat conda~ and receive the following output:

[DEFAULT]

name = conda
brief = Conda - package, dependency, and environment management.
url = https://conda.io/docs/
category = environments

rootdir = /opt/browncis/%(name)s/envs/%(version)s
prepend PATH = %(rootdir)s/bin

[root]
default = true
rootdir = /opt/browncis/%(name)s
prepend PATH = %(rootdir)s/bin

[stronghold_v1]

[bcbi_v1]

[psteytest]

Since I don't have write permissions to /opt/stronghold/pymodules, I'm going to have to figure something else out. I'm going to make my own pymodules set up and use that instead of the one in opt/stronghold.

First, let's figure out what my home directory is. I run expand ~ and receive the following output:

expand: /data-home/daluthge: Is a directory

Next, I run the following command: cp -R /opt/stronghold/pymodules /data-home/daluthge/dilum-pymodules

Now, I run vim /data-home/daluthge/dilum-pymodules/init/modules.sh. Vim opens. I edit lines 28 and 29 so they look like this:

export MODULEHOME=/data-home/daluthge/dilum-pymodules
export MODULEPATH=/data-home/daluthge/dilum-pymodules/modulefiles

I save the file and quit Vim.

Now, I'm going to set up .bash_profile, .bashrc, .profile, and .juliarc.jl. I set them up as follows:

.bash_profile:

if [ -f ~/.profile ]; then
    source ~/.profile
fi

if [ -f ~/.bashrc ]; then
    source ~/.bashrc
fi

.bashrc:

source /data-home/daluthge/dilum-pymodules/init/modules.sh
moduledb rebuild

export JULIA_LOAD_CACHE_PATH=/data-home/daluthge/.julia_cache/

.profile:

.juliarc.jl:

Base.LOAD_CACHE_PATH = ENV["JULIA_LOAD_CACHE_PATH"]

Finally, I run vim /data-home/daluthge/dilum-pymodules/modulefiles/conda. Vim opens. I add two new lines at the end of the file with the following contents:

[bcbi_v0.0.1]
set JULIA_PKGDIR = %(rootdir)s/lib/julia/packages

I close Terminal. Now, I open a new Terminal. I type module load conda/bcbi_v0.0.1 and I get the following output:

module: loading 'conda/bcbi_v0.0.1'

I type Julia and press enter. Now I am in a Julia REPL. I type import PredictMD and I get the message INFO: Precompiling module PredictMD.

I am still waiting for the precompilation to finish. I will post an update when it is done.

[DilumAluthge]

I can confirm that julia -e 'import PredictMD' runs successfully, i.e. PredictMD precompiles successfully.

[mirestrepo]

I'm going to close this, since this is out of my powers. You should create a ticket by sending an email to stronghold-helpATbrown.edu. That way the issue can be addressed by those who have the right permissions

[DilumAluthge]

@mirestrepo Just emailed stronghold-help.

[DilumAluthge]

The command module load conda/bcbi_v0.0.1 now successfully runs for most Stronghold users.