Rocoto on GaeaC6

[BijuThomas-NOAA]

I installed Rocoto on Gaea C6. Installation was successful but got the following error with rocotostat:

/ncrc/proj/hurr1/hafs/shared/rocoto/dev/bin/rocotostat -w hafs-hafs_dev062024_hfsb_dev-09L-2021082712.xml -d hafs-hafs_dev062024_hfsb_dev-09L-2021082712.dbTraceback (most recent call last):
12: from /ncrc/proj/hurr1/hafs/shared/rocoto/dev/bin/../sbin/rocotostat.rb:17:in `<main>'
11: from /usr/lib64/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
10: from /usr/lib64/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
 9: from /autofs/ncrc-svm1_proj/hurr1/hafs/shared/rocoto/dev/lib/wfmstat/statusengine.rb:6:in `<top (required)>'
8: from /autofs/ncrc-svm1_proj/hurr1/hafs/shared/rocoto/dev/lib/wfmstat/statusengine.rb:13:in `<module:WFMStat>'
7: from /autofs/ncrc-svm1_proj/hurr1/hafs/shared/rocoto/dev/lib/wfmstat/statusengine.rb:17:in `<class:StatusEngine>'
6: from /usr/lib64/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
 5: from /usr/lib64/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
4: from /autofs/ncrc-svm1_proj/hurr1/hafs/shared/rocoto/dev/lib/workflowmgr/workflowdb.rb:6:in `<top (required)>'
3: from /autofs/ncrc-svm1_proj/hurr1/hafs/shared/rocoto/dev/lib/workflowmgr/workflowdb.rb:37:in `<module:WorkflowMgr>'
 2: from /autofs/ncrc-svm1_proj/hurr1/hafs/shared/rocoto/dev/lib/workflowmgr/workflowdb.rb:39:in `<class:WorkflowSQLite3DB>'
1: from /usr/lib64/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
/usr/lib64/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- sqlite3 (LoadError)

Installation was with the default modules, listed here: module list

Currently Loaded Modules:
1) craype-x86-genoa     4) perftools-base/23.12.0                 7) craype/2.7.30      10) cray-libsci/24.03.0  13) darshan-runtime/3.4.0  16) DefApps/default
2) libfabric/1.20.1     5) xpmem/2.8.2-1.0_5.1__g84a27a5.shasta   8) cray-dsmml/0.2.2   11) PrgEnv-intel/8.5.0   14) CmrsEnv/default
3) craype-network-ofi   6) intel/2023.2.0                         9) cray-mpich/8.1.28  12) cray-pmi/6.1.13      15) TimeZoneEDT/default

ruby -v
ruby 2.5.9p229 (2021-04-05 revision 67939) [x86_64-linux-gnu]
irb
irb(main):001:0> require 'sqlite3'
Traceback (most recent call last):
        4: from /usr/bin/irb:11:in `<main>'
        3: from (irb):1
        2: from /usr/lib64/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
        1: from /usr/lib64/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
LoadError (cannot load such file -- sqlite3)

Is this due to an issue with sqlite3-ruby installation on Gaea?

[christopherwharrop-noaa]

A couple things. First, this error indicates the install was not successful. When you ran ./INSTALL did it finish without errors? Second, you should not install Rocoto with any compiler other than gcc. You should unload Intel and replace it with gnu.

[christopherwharrop-noaa]

Or, if gcc is on the path, you could try setting CC=gcc before doing the install to make sure Intel compilers are not used to build anything. While Intel should work, it isn't needed, and you're usually better off not using it.

[BijuThomas-NOAA]

Thanks @christopherwharrop-noaa , build with gcc and I see this errors now:

checking for sqlite3.h... no
Could not find sqlite3.h.
Please visit https://github.com/sparklemotion/sqlite3-ruby for installation instructions.

*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

ls -l /ncrc/proj/hurr1/hafs/shared/rocoto/dev/lib/sqlite3/lib total 2436

-rw-r--r-- 1 Biju.Thomas ncep 1325374 Aug 12 11:37 libsqlite3.a
-rwxr-xr-x 1 Biju.Thomas ncep    1023 Aug 12 11:37 libsqlite3.la*
lrwxrwxrwx 1 Biju.Thomas ncep      19 Aug 12 11:37 libsqlite3.so -> libsqlite3.so.0.8.6*
lrwxrwxrwx 1 Biju.Thomas ncep      19 Aug 12 11:37 libsqlite3.so.0 -> libsqlite3.so.0.8.6*
-rwxr-xr-x 1 Biju.Thomas ncep 1141656 Aug 12 11:37 libsqlite3.so.0.8.6*
drwxr-xr-x 2 Biju.Thomas ncep    4096 Aug 12 11:37 pkgconfig/

The build log is here on Gaea C6: /ncrc/proj/hurr1/hafs/shared/rocoto/dev/build.log

[christopherwharrop-noaa]

This means the system Ruby does not have the ruby-devel OS packages installed. You will need to build your own Ruby, or find one that someone else has already installed, and then use ./INSTALL --with-ruby=/path/to/ruby.

[BijuThomas-NOAA]

Thanks Chris. I reached out to Gaea Help desk and they would like to know:

Will rocoto need these header files on the compute nodes, or only the C6 login nodes? Also are there other header files that are required for rocoto? Reach out to the rocoto development team for information on the required packages

[christopherwharrop-noaa]

Headers for ruby are required. Rocoto's current installation method bundles its own libxml2 and sqlite3 installations, so those should not be necessary. That is likely to change in the future, however, which will mean that libxml2 and sqlite3 headers/libs will be needed as well in a future release.

However, for now, the easiest thing to do is to just install your own Ruby. It's an extremely simple install and you don't need to worry about involving admins. Just make sure you use GNU compilers for the build, and not Intel.

Follow these steps:

#!/usr/bin/env bash

set -e

# Install ruby
wget https://cache.ruby-lang.org/pub/ruby/3.2/ruby-3.2.3.tar.gz
tar -xzvf ruby-3.2.3.tar.gz
pushd ruby-3.2.3
./configure --prefix=$HOME/opt/ruby/3.2.3 --disable-install-doc
make
make install
popd
rm -f ruby-3.2.3.tar.gz

# Install Rocoto
mkdir -p $HOME/opt/rocoto/1.3.7
pushd $HOME/opt/rocoto/1.3.7
wget https://github.com/christopherwharrop/rocoto/archive/refs/tags/1.3.7.tar.gz
tar --strip-components=1 -xzvf 1.3.7.tar.gz
./INSTALL --with-ruby=$HOME/opt/ruby/3.2.3
rm -rf 1.3.7.tar.gz

[christopherwharrop-noaa]

I forgot to answer your other question. The ruby-devel package that provides headers is only needed on service or login nodes when installing Rocoto. It is not needed on compute nodes.

[BijuThomas-NOAA]

Thank you very much. Installation is successful after following your instructions and Rocoto is working fine on C6

[christopherwharrop-noaa]

Glad to hear it @BijuThomas-NOAA.