search

Bright-Computing / bic

Bright-Illumina collaboration
GNU General Public License v2.0
4 stars 5 forks source link

Break down Bright modules namespace on per category depth level, for Lmod compatibility #4

Closed fgeorgatos closed 7 years ago

fgeorgatos commented 8 years ago

as discussed, these should belong to distinct directories:

    category=0            category=1         category=2
    ----------            ----------         --------------------------

    Intel-mpi/64/5.0/4    cuda70/blas/7.0    bio/genomics/bowtie/64/3.2
    Intel-mpi/mic/5.0/4   cuda70/fft/7.0     bio/genomics/tophat/64/4.7

Then, the only expected change would be the output of the command module avail. fi. MODULEPATH would have 2 or 3 sections instead of one. Everything else we expect it to work like before, without any changes at all.

Why we do this? Lmod would then be able to honor the one-name rule! (protects from conflicts)

fgeorgatos commented 8 years ago

@rtmclay : Here is the current layout ; the section called modulefiles2 may require some further effort, no?

[root@bright-lmod shared]# hostname -f
bright-lmod.cm.cluster
[root@bright-lmod shared]# echo $MODULEPATH
/cm/local/modulefiles0:/cm/local/modulefiles1:/cm/shared/modulefiles0:/cm/shared/modulefiles1:/cm/shared/modulefiles2:/root/privatemodules
[root@bright-lmod shared]# module av

----------------------------------------------- /cm/local/modulefiles0 -----------------------------------------------
   cmd (L)    cmsh (L)    dot    module-git    module-info    null    openldap    shared (L
)    use.own    version

----------------------------------------------- /cm/local/modulefiles1 -----------------------------------------------
   cluster-tools/7.1 (L,D)    cm-scale-cluster/7.1 (D)    freeipmi/1.4.8 (D)    gcc/5.1.0 (D)    ipmitool/1.8
.15 (D)

---------------------------------------------- /cm/shared/modulefiles0 -----------------------------------------------
   bonnie++/1.97.1 (D)      hpl/2.1                     (D)    netperf/2.6.0        (D)    sge/2011.11p1 (D)
   cmgui/7.1       (L,D)    hwloc/1.9.1                 (D)    open64/4.5.2.1       (D)    slurm/14.11.6 (
L,D)
   gdb/7.9         (D)      intel-cluster-checker/2.2.2 (D)    openlava/3.0         (D)    torque/5.1.0  (D)
   hdf5/1.6.10     (D)      iozone/3_430                (D)    pbspro/13.0.0.151487 (D)
   hdf5_18/1.8.14  (D)      mpiexec/0.84_432            (D)    puppet/3.7.5         (D)

---------------------------------------------- /cm/shared/modulefiles1 -----------------------------------------------
   blas/gcc/64/3.5.0                 (D)    intel-mpi/64/5.0.3/048               (D)
   blas/open64/64/3.5.0              (D)    intel-mpi/mic/5.0.3/048              (D)
   intel-cluster-runtime/ia32/3.7    (D)    intel-tbb-oss/ia32/43_20150424oss    (D)
   intel-cluster-runtime/intel64/3.7 (D)    intel-tbb-oss/intel64/43_20150424oss (D)
   intel-cluster-runtime/mic/3.7     (D)    openblas/dynamic/0.2.14              (D)

---------------------------------------------- /cm/shared/modulefiles2 -----------------------------------------------
   acml/gcc-int64/64/5.3.1         (D)    acml/open64/mp/fma4/5.3.1            (D)    mpich/ge/open64/64/3.1.4  (D)
   acml/gcc-int64/fma4/5.3.1       (D)    blacs/openmpi/gcc/64/1.1patch03      (D)    mvapich/gcc/64/1.2rc1     (D)
   acml/gcc-int64/mp/64/5.3.1      (D)    blacs/openmpi/open64/64/1.1patch03   (D)    mvapich/intel/64/1.2rc1   (D)
   acml/gcc-int64/mp/fma4/5.3.1    (D)    fftw2/openmpi/gcc/64/double/2.1.5    (D)    mvapich/open64/64/1.2rc1  (D)
   acml/gcc/64/5.3.1               (D)    fftw2/openmpi/gcc/64/float/2.1.5     (D)    mvapich2/gcc/64/2.1       (D)
   acml/gcc/fma4/5.3.1             (D)    fftw2/openmpi/open64/64/double/2.1.5 (D)    mvapich2/intel/64/2.1     (D)
   acml/gcc/mp/64/5.3.1            (D)    fftw2/openmpi/open64/64/float/2.1.5  (D)    mvapich2/open64/64/2.1    (D)
   acml/gcc/mp/fma4/5.3.1          (D)    fftw3/openmpi/gcc/64/3.3.4           (D)    netcdf/gcc/64/4.3.3.1
   acml/open64-int64/64/5.3.1      (D)    fftw3/openmpi/open64/64/3.3.4        (D)    netcdf/open64/64/4.3.3.1
   acml/open64-int64/fma4/5.3.1    (D)    globalarrays/openmpi/gcc/64/5.3      (D)    openmpi/gcc/64/1.8.5      (D)
   acml/open64-int64/mp/64/5.3.1   (D)    globalarrays/openmpi/open64/64/5.3   (D)    openmpi/intel/64/1.8.5    (D)
   acml/open64-int64/mp/fma4/5.3.1 (D)    intel/compiler/64/15.0/2015.5.223    (D)    openmpi/open64/64/1.8.5   (D)
   acml/open64/64/5.3.1            (D)    lapack/gcc/64/3.5.0                  (D)    scalapack/gcc/64/1.8.0    (D)
   acml/open64/fma4/5.3.1          (D)    lapack/open64/64/3.5.0               (D)    scalapack/open64/64/1.8.0 (D)
   acml/open64/mp/64/5.3.1         (D)    mpich/ge/gcc/64/3.1.4                (D)

------------------------------------------------ /root/privatemodules ------------------------------------------------
   null
rtmclay commented 8 years ago

I think that modulefile2 is mostly O.K. I think that there is a possible problem with acml/gcc-int64/64/5.3.1 and acml/gcc/64/5.3.1. According to the rules that we are proposing a user could load both acml/gcc-int64/... and acml/gcc. If that is O.K. then I guess things would be O.K.

comments?

fgeorgatos commented 8 years ago

Wouldn't "acml" be the conflicting part? If not, this issue would exist with all others, no?

On Wednesday, 30 March 2016, Robert McLay notifications@github.com wrote:

I think that modulefile2 is mostly O.K. I think that there is a possible problem with acml/gcc-int64/64/5.3.1 and acml/gcc/64/5.3.1. According to the rules that we are proposing a user could load both acml/gcc-int64/... and acml/gcc. If that is O.K. then I guess things would be O.K.

comments?

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/plabrop/bic/issues/4#issuecomment-203597509

echo "sysadmin know better bash than english"|sed s/min/mins/ \ | sed 's/better bash/bash better/' # signal detected in a CERN forum

fgeorgatos commented 8 years ago

@rtmclay : any news on supporting multiple categories, yet?! (any hint to likely ETAs might be handy)

rtmclay commented 8 years ago

No ETA. I have started working on it but every time I start looking at it my head spins. It is basically a rewrite of Lmod from the ground up. Eventually, I'll be able to reuse part of the current implementation but until I'm farther along I can't give an ETA. I hope to have it done by the end of this year, but I'm not promising that.

fgeorgatos commented 8 years ago

@rtmclay : ack that; an effort to fix the namespace on the Bright side might be the easy way out nevertheless

fgeorgatos commented 7 years ago

since 7.x Lmod release things work as they should and there is no immediate need for category split; therefor I consider this case closed for now - to be reopened if reality proves need to do otherwise.