Address dev/prod parity issues for AweSim developers

[ericfranz]

Our current plan is for AweSim developers to develop and build apps on an Oakley login node. To help with parity between dev (Oakley login node) and prod (web02) SCL was installed on Oakley with the same packages as are installed on the web hosts. However there are other parity issues that make using an Oakley login node for development problematic.

1. missing cluster config - apps using OodAppkit now rely in the presence of a cluster config directory with a default location at /etc/ood/config/clusters.d in order to build a list of available clusters, their servers (Torque, Moab, Ganglia) and the paths to shared libraries used to interact with them
2. the required cluster config will differ on Oakley - on web02 the Torque libs are installed at /opt/torque/lib64 whereas on Oakley they are installed at /usr/local/torque/default so we will need to maintain two different cluster configs if we have developers use Oakley login nodes for development
3. Oakley login nodes have Torque 5 but web service hosts have Torque 6 - so I won't be able to submit jobs to Owens from the login node. Now most of the time I will be making code modifications and testing the app via the web (running on a web service node). However, if I try to test using the Rails console or running scripts this disparity could cause problems.

Is that the full list of disparity between the Oakley login node and the webservice hosts?

[nickjer]

• need to be able to submit jobs to all clusters that you can submit to from corresponding web node for testing (currently can't submit to owens or ruby from oakley)
• need to clone /var/www/ood from web node to oakley login node (as some "public" assets may be necessary for rails environment to run, whether in console or precompiling assets), i.e., Total Sim wants to manipulate the publicly available header png and serve it from within the app (made this up, but it is a possibility)
• if the above is true, then which /var/www/ood/public do we copy? ondemand or awesim?
• if we provide portal specific configs in the future, which do we copy over?

[ericfranz]

The ruby scl packages differ on Oakley login node and webservice host.

webdev02 has bundler, bundle, (and thor) executables:

┌─[jnicklas@webdev02][~/ondemand/dev/ood-dashboard][±][master ✓]
└─▪ ll /opt/rh/rh-ruby22/root/usr/bin/
total 56
dr-xr-xr-x  2 root root 4096 Aug 10 15:47 .
drwxr-xr-x 13 root root 4096 Aug 10 15:46 ..
-rwxr-xr-x  1 root root  514 Feb  5  2015 bundle
-rwxr-xr-x  1 root root  515 Feb  5  2015 bundler
-rwxr-xr-x  1 root root 4860 Jun 24 10:52 erb
-rwxr-xr-x  1 root root  564 Jun 24 10:52 gem
-rwxr-xr-x  1 root root  208 Jun 24 10:52 irb
-rwxr-xr-x  1 root root 1262 Jun 24 10:52 rake
-rwxr-xr-x  1 root root  956 Jun 24 10:52 rdoc
-rwxr-xr-x  1 root root  206 Jun 24 10:52 ri
-rwxr-xr-x  1 root root 5344 Jun 24 10:53 ruby
-rwxr-xr-x  1 root root  503 Jan  9  2015 thor

oakley does not:

┌─[jnicklas@oakley02][~/ondemand/dev/test][±][master U:2 ✗]
└─▪ ll /opt/rh/rh-ruby22/root/usr/bin/
total 44
dr-xr-xr-x  2 root root 4096 Aug  4 16:02 .
drwxr-xr-x 13 root root 4096 Aug  4 15:59 ..
-rwxr-xr-x  1 root root 4860 Jun 24 10:52 erb
-rwxr-xr-x  1 root root  564 Jun 24 10:52 gem
-rwxr-xr-x  1 root root  208 Jun 24 10:52 irb
-rwxr-xr-x  1 root root 1262 Jun 24 10:52 rake
-rwxr-xr-x  1 root root  956 Jun 24 10:52 rdoc
-rwxr-xr-x  1 root root  206 Jun 24 10:52 ri
-rwxr-xr-x  1 root root 5344 Jun 24 10:53 ruby

[ericfranz]

need to be able to submit jobs to all clusters that you can submit to from corresponding web node for testing (currently can't submit to owens or ruby from oakley)

It is true that many developers will typically be targeting only one cluster with their apps (Oakley for starters). But our apps target support for clusters so if we can't submit to all clusters from Oakley we won't use Oakley to develop the dashboard, or any of the accessibility apps.

Its important that WIAG developers use the same development nodes as other developers so that we address any problems with the dev environment and others benefit.

[ericfranz]

Dev's use webtest02