Install ASM Issue

[Mohammed-Bhatti]

Hi Edwin,

I'm trying to install 11gR2 on RHEL 6.5 using oradb::installasm.

I have gridBase => '/app/oragrid/base' and gridHome => '/app/oragrid/11.2.0/grid'

When I run the agent, I get the following error from oradb::instalasm: Error: Could not retrieve catalog from remote server: Error 400 on SERVER: gridHome folder should be under the gridBase folder at /etc/puppet/modules/oradb/manifests/installasm.pp:65

Yet, when I put the gridHome under gridBase: gridBase => '/app/oragrid/base', gridHome => '/app/oragrid/base/11.2.0/grid',

Then I get a runInstaller error:

Notice: /Stage[main]/Test01::Createasm/Oradb::Installasm[asm]/Exec[install oracle grid asm]/returns: Preparing to launch Oracle Universal Installer from /tmp/OraInstall2015-02-23_06-22-00PM. Please wait ...[FATAL] [INS-32026] The Software Location specified should not be under Oracle base location.

Notice: /Stage[main]/Test01::Createasm/Oradb::Installasm[asm]/Exec[install oracle grid asm]/returns: CAUSE: Grid Infrastructure for a Cluster installation will assign root ownership to all parent directories of Oracle Clusterware software location. As a result, all named directories in the software location path will acquire root ownership. This may create problems for subsequent installations into the same Oracle base.

Notice: /Stage[main]/Test01::Createasm/Oradb::Installasm[asm]/Exec[install oracle grid asm]/returns: ACTION: Specify software location outside of Oracle base.

Seems a bit like a catch-22 situation here. Am I doing something wrong?

Regards,

mohammed

[biemond]

Hi Mohammed,

that's strange, I do the same in this vagrant box.

See this https://github.com/biemond/biemond-oradb-vagrant-11.2-ASM/blob/master/puppet/manifests/site.pp and the matching hiera values https://github.com/biemond/biemond-oradb-vagrant-11.2-ASM/blob/master/puppet/hieradata/dbasm.example.com.yaml

maybe you need to set the oraInventoryDir to /app and use for the oracle database base ( not grid) to /app/oracle and oracle home to /app/oracle/product/...

Hope this helps

[Mohammed-Bhatti]

Hi Edwin,

Thanks for sharing. I'm not using hiera (at the moment), so I have a class defined as follows:

class test01::createasm() {

oradb::installasm { 'asm': version => '11.2.0.4', file => 'p13390677_112040_Linux-x86-64_3of7.zip', gridType => 'CRS_CONFIG', gridBase => '/app/grid', gridHome => '/app/grid/product/11.2/grid', oraInventoryDir => '/app', user => 'oragrid', userBaseDir => '/home', group => 'oinstall', group_install => 'oinstall', group_oper => undef, group_asm => 'asmadmin', sys_asm_password => 'Passw0rd', asm_monitor_password => 'Passw0rd', asm_diskgroup => 'DATA', disk_discovery_string => '/dev/mapper/*', disk_redundancy => 'EXTERNAL', disks => '/dev/mapper/shared', downloadDir => '/app/install', puppetDownloadMntPoint => '/app/install', remoteFile => false, zipExtract => true, cluster_name => 'foobar-cluster', scan_name => 'foobar-cluster-scan', scan_port => 1521, cluster_nodes => 'node01.foo.bar:node01-vip.foo.bar,node02.foo.bar:node02-vip.foo.bar', network_interface_list => 'eth0:10.10.122.0:1,eth3:10.10.11.0:2,eth1:172.20.2.0:3', storage_option => 'ASM_STORAGE', } }

Strange that I'm getting this error. Again, I'm using RHEL 6.5 and just trying to install GI at the moment and no database components at this time.

Perhaps I need to tweak the exec statements?

Thanks

mohammed

[oravirt]

Hi Mohammed,

When doing an install of gridType='CRS_CONFIG' (cluster install), your gridHome can not be under gridBase. If this is a standalone GI install that is not a problem.

You have: gridBase=/app/grid gridHome='/app/grid/product/11.2/grid' I think you have to put gridBase=/app/oracle gridHome='/app/grid/product/11.2/grid'

regards /Mikael

[biemond]

Thanks,

you are right because the oracle grid home is probably shared and should be on a shared disc. Oracle grid base will contain all the node specific DB/ASM information.

I will change it

Thanks

[biemond]

Done, pushed a new version to forge and github

[Mohammed-Bhatti]

Thanks for you input Mikael.

Edwin,

Thanks for the update. I downloaded the latest version and ran it and had better luck. However, node2 didn't complete the full software install. CRS came up on node1. We were having some storage issues which have not been completely resolved which I most likely impacted the install.

I'm waiting for these issues to be resolved and will re-test as soon as I can and post an update.

Thanks again.

mohammed

[Mohammed-Bhatti]

Hi, So here's an update. I tried to do a 3-node install. The issue I'm having now is that the software copy on the other two nodes does not seem to complete. It looks like the software is there on the other nodes but important bits are missing e.g. /app/oragrid/12.1.0/grind/bin/crsctl. The crsctl.bin file is there on both nodes but not the crsctl file. This is only available on node1. So the cluster configuration on the other two nodes never completes.

I don't see any errors in the install logs either.

Thanks

mohammed

[biemond]

Hi,

Are you doing the same install on all 3 nodes or do you do a software copy? And can I see your puppet manifest. Thanks

[Mohammed-Bhatti]

Yes, doing the same install on all 3 nodes. Not a software copy.

I did try the CRS_SWONLY option on all three nodes and I do notice that crsctl does not exist on any of the three nodes which leads me to believe that this only gets built during the configuration phase. As as test, I ran the runInstaller GUI and did a software only install. The crsctl file also does not exist. I'm sure there are a bunch of files that only get built during the configuration phase only.

So I guess I'm trying to figure out why the configuration does not complete on the other two nodes when using the CRS_CONFIG option.

Here is my site.pp:

node node1, node2, node3 {
   class { "test01::createasm12c": }
}

And the class that it calls:

class test01::createasm12c() {
   oradb::installasm { 'asm':
      version           =>      '12.1.0.1',
      file              =>      'V46096-01',
      gridType          =>      'CRS_CONFIG',
      gridBase          =>      '/app/oragrid/base',
      gridHome          =>      '/app/oragrid/12.1.0/grid',
      oraInventoryDir   =>      '/app',
      user              =>      'oragrid',
      userBaseDir       =>      '/home',
      group             =>      'oinstall',
      group_install     =>      'oinstall',
      group_oper        =>      'asmdba',
      group_asm         =>      'asmadmin',
      sys_asm_password  =>      'Welcome1',
      asm_monitor_password      =>      'Welcome1',
      asm_diskgroup     =>      'DATA1',
      disk_discovery_string     =>      '/dev/asm*',
      disk_redundancy   =>      'EXTERNAL',
      disks             =>      '/dev/asm-disk1',
      downloadDir       =>      '/app/install',
      puppetDownloadMntPoint    =>      '/app/install',
      remoteFile                =>      false,
      zipExtract        =>      true,
      cluster_name      =>      'node-cluster',
      scan_name         =>      'node-scan',
      scan_port         =>      1521,
      cluster_nodes     =>      'node1.foo.bar:node1-vip.foo.bar,node2.foo.bar:node2-vip.foo.bar,node3.foo.bar:node3-vip.foo.bar',
      network_interface_list    =>      'eth1:10.10.112.0:1,eth2:192.168.113.0:2,eth0:192.168.108.0:2',
      storage_option    =>      'ASM_STORAGE',
   }
}

Thanks

mohammed

[biemond]

Hi

When you want to create a non rac database on ASM you can use this.

For RAC you should do it once and put ASM instance on a shared storage and the other nodes should clone the oracle software. Else you have will have 3 ASM instances instead of 1 with 3 nodes.

here you have a complete oracle RAC example on centos 6.5 vagrant box https://github.com/hajee/vagrant-centos-6.5-ora11-rac it with uses oradb::installasm and the ora_rac module of Bert Hajee

I will do some more research, hope this helps.

thanks Edwin

[Mohammed-Bhatti]

Hi Edwin,

Thanks for the update. Yes, I've looked at Bert's module.

They way I ended up doing this on a 3-node 12c GI install, was to install the software only on all nodes and then just run $GI_HOME/crs/config/config.sh with the generated response file from your module. The only change I made to your module was to comment out ${gridHome}/crs/install/roothas.pl (don't want to config HA services, let config.sh do it) so that config.sh can complete successfully.

Your module (along with Bert's) have been very helpful in automating the GI install.

Thanks

mohammed

[biemond]

Ok cool,

so what can I do to fix or help you with this, because roothas.pl is only executed when $gridType == 'CRS_SWONLY'

[Mohammed-Bhatti]

Maybe an option to just install the software without running any config scripts? If I'm not mistaken, if the roothas.pl script is run on each node, each node then becomes an independent node and not part of a cluster. If this script is not run, we can use config.sh and the response file created by installasm.pp module to configure ASM using config.sh.

BTW, just to be clear, I used oradb::installasm to install the GI software using the 'CRS_SWONLY' option on all three nodes but modified it so that roothas.pl was not run. To be able to this is a huge step for us.

Also taking a look at Bert's ora_rac module as you suggested and that seems to answer a lot of questions I have. We're constrained to using 3.4.2 whereas ora_rac is written for 3.7.x so this module wouldn't work for us without changes.

mohammed

[biemond]

Done in the latest commit e03bdba67bad6e55de7d0c12f59e99aee41e16c1 of oradb you can now set $stand_alone parameter true or false in combination with $gridType == 'CRS_SWONLY'

Thanks

[Mohammed-Bhatti]

Thanks for the update:

Got the following error:

Error: /bin/sh: /app/oragrid/12.1.0/grid/cfgtoollogs/configToolAllCommands: No such file or directory

Error: /Stage[main]/Test01::Createasm12c/Oradb::Installasm[asm]/Exec[run configToolAllCommands grid tool asm]/returns: change from notrun to 0 3 failed: /bin/sh: /app/oragrid/12.1.0/grid/cfgtoollogs/configToolAllCommands: No such file or directory

Here's part of the manifest: version => '12.1.0.1', file => 'V46096-01', gridType => 'CRS_SWONLY',

gridType => 'CRS_CONFIG',

  stand_alone       =>      false,
  gridBase          =>      '/app/oragrid/base',
  gridHome          =>      '/app/oragrid/12.1.0/grid',
  oraInventoryDir   =>      '/app',

I also tried stand_alone => true which worked with no problems.

[biemond]

got it , made a stupid mistake so it always end up in the else branch