search

biemond / biemond-orawls

Puppet 3/4 module for WebLogic provisioning module ( Linux & Solaris )
Apache License 2.0
63 stars 104 forks source link

timeout creating wls_user #419

Closed rssh22 closed 7 years ago

rssh22 commented 7 years ago

Hi,

I've followed the next sequence of creation events:

  1. create the /etc/wls_settings.yaml
  2. install jdk
  3. install middleware home (/middleware/wld-pro-04/wl1036/)
  4. create a domain (wld-pro-04)
  5. install nodemanager
  6. create the control class

in step 6. create a wls_user i'm getting a timeout error. 

Debug: Executing: '/usr/bin/svcs -H -o state,nstate mcollective'
Debug: Executing: '/usr/bin/svcs -H -o state,nstate mcollective'
Debug: Exec[mkdir_p_/middleware/wld-pro-04/](provider=posix): Executing check 'test -d '/middleware/wld-pro-04/''
Debug: Executing: 'test -d '/middleware/wld-pro-04/''
Debug: Exec[mkdir_p_/middleware/wld-pro-04/_chmod__0755](provider=posix): Executing check 'test $(stat -c %a '/middleware/wld-pro-04/') = 0755 -o $(stat -c %a '/middleware/wld-pro-04/') = 755'
Debug: Executing: 'test $(stat -c %a '/middleware/wld-pro-04/') = 0755 -o $(stat -c %a '/middleware/wld-pro-04/') = 755'
Debug: Exec[mkdir_p_/middleware/wld-pro-04/_chown__bea](provider=posix): Executing check 'test $(stat -c %U '/middleware/wld-pro-04/') = bea -o $(stat -c %u '/middleware/wld-pro-04/') = bea'
Debug: Executing: 'test $(stat -c %U '/middleware/wld-pro-04/') = bea -o $(stat -c %u '/middleware/wld-pro-04/') = bea'
Debug: Exec[mkdir_p_/middleware/wld-pro-04/_chgrp__bea](provider=posix): Executing check 'test $(stat -c %G '/middleware/wld-pro-04/') = bea -o $(stat -c %g '/middleware/wld-pro-04/') = bea'
Debug: Executing: 'test $(stat -c %G '/middleware/wld-pro-04/') = bea -o $(stat -c %g '/middleware/wld-pro-04/') = bea'
Debug: Exec[startNodemanager ndmgr-wld-pro-04](provider=posix): Executing check '/bin/ps wwxa | /bin/grep -v grep | /bin/grep 'weblogic.NodeManager''
Debug: Executing with uid=bea gid=bea: '/bin/ps wwxa | /bin/grep -v grep | /bin/grep 'weblogic.NodeManager''
Debug: /Stage[main]/Fussion::Create::Nodemanager/Orawls::Nodemanager[ndmgr-wld-pro-04]/Exec[startNodemanager ndmgr-wld-pro-04]/unless:  20427 ?        S  0:06 /var/tmp/java-1.6.0_131/bin/sparcv9/java -jrockit -Xms128m -Xmx256m -Dcoherence.home=/middleware/wld-pro-04/wl1036/coherence_3.7 -Dbea.home=/middleware/wld-pro-04/wl1036 -Dweblogic.ssl.JSSEEnabled=false -Dweblogic.security.SSL.enableJSSE=false -Xverify:none -Djava.security.policy=/middleware/wld-pro-04/wl1036/wlserver/server/lib/weblogic.policy -Dweblogic.nodemanager.javaHome=/var/tmp/java-1.6.0_131 weblogic.NodeManager -v
Debug: adminserver_status /bin/ps awwx | /bin/grep -v grep | /bin/grep 'weblogic.Name=wls-pro-04-sa-adm-n0' | /bin/grep wld-pro-04
Debug: line  20490 ?        S  0:58 /var/tmp/java-1.6.0_131/bin/sparcv9/java -jrockit -Xms512m -Xmx512m -Dweblogic.Name=wls-pro-04-sa-adm-n0 -Djava.security.policy=/middleware/wld-pro-04/wl1036/wlserver/server/lib/weblogic.policy -Dweblogic.ProductionModeEnabled=true -Dweblogic.system.BootIdentityFile=/domains/wld-pro-04/servers/wls-pro-04-sa-adm-n0/security/boot.properties -Dweblogic.nodemanager.ServiceEnabled=true -XX:PermSize=256m -XX:MaxPermSize=512m -Xms1024m -Xmx1024m -Dweblogic.Stdout=/domains/wld-pro-04/servers/wls-pro-04-sa-adm-n0/logs/wls-pro-04-sa-adm-n0.out -Dweblogic.Stderr=/domains/wld-pro-04/servers/wls-pro-04-sa-adm-n0/logs/wls-pro-04-sa-adm-n0_err.out -da -Dplatform.home=/middleware/wld-pro-04/wl1036/wlserver -Dwls.home=/middleware/wld-pro-04/wl1036/wlserver/server -Dweblogic.home=/middleware/wld-pro-04/wl1036/wlserver/server -Dweblogic.management.discover=true -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole=false -Dweblogic.ext.dirs=/middleware/wld-pro-04/wl1036/patch_wls1036/profiles/default/sysext_manifest_classpath:/middleware/wld-pro-04/wl1036/patch_ocp371/profiles/default/sysext_manifest_classpath weblogic.Server

Debug: found server
Debug: adminserver_status output Found
Debug: Prefetching simple resources for wls_user
Debug: index wls_user
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Debug: Dynamically-bound server lookup failed, falling back to server setting
Debug: Dynamically-bound port lookup failed; falling back to masterport setting
Debug: Using cached connection for https://vm-lab-linux-1.msc.es:8140
Debug: Caching connection for https://vm-lab-linux-1.msc.es:8140
Debug: Executing: wlstScript with action index
Debug: domain found wld-pro-04
Prepare to run: /tmp/wlstScript20170308-20535-zca9j6.py,bea,wld-pro-04,/middleware/wld-pro-04/wl1036/wlserver,weblogic,welcome1,t3://puppet-pil00.msc.es:10400
vvv===================================================================

from weblogic.management.security.authentication import UserReaderMBean

realmName    = cmo.getSecurityConfiguration().getDefaultRealm()
authProvider = realmName.getAuthenticationProviders()

f = open_file("name;realm;authenticationprovider;description;domain")
for i in authProvider:
  if isinstance(i,UserReaderMBean) and "ldap" not in i.getProviderClassName().lower():
    userName = i
    authName = i.getName()

    cursor = i.listUsers('*',0)
    while userName.haveCurrent(cursor):
      user = userName.getCurrentName(cursor)
      description = i.getUserDescription(user)

      add_index_entry(f, [domain+'/'+user, realmName.getName(), authName,description,domain])

      userName.advance(cursor)

    userName.close(cursor)

f.close()
report_back_success()

^^^====================================================================
Debug: no custom trust
Info: Starting the wls daemon for domain wld-pro-04
Debug: Defining common methods...
Debug: Dynamically-bound server lookup failed, falling back to server setting
Debug: Dynamically-bound port lookup failed; falling back to masterport setting
Debug: Using cached connection for https://vm-lab-linux-1.msc.es:8140
Debug: Caching connection for https://vm-lab-linux-1.msc.es:8140
Info: Executing wls-script /tmp/wlstCommonScript.py20170308-20535-1k7lj83
Info: Executing wls-script /tmp/wlstScript20170308-20535-zca9j6.py with timeout = 120
Debug: Passing domain wld-pro-04
Debug: Passing use_default_value_when_empty False
Debug: Passing credentials to WLST
Info: Connecting to wls on url t3://puppet-pil00.msc.es:10400
Debug: Oracle Corporation SunOS 5.11      11.2    June 2014 [00]

Debug: CLASSPATH=/middleware/wld-pro-04/wl1036/patch_wls1036/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/middleware/wld-pro-04/wl1036/patch_ocp371/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/usr/jdk/instances/jdk1.7.0/lib/tools.jar:/middleware/wld-pro-04/wl1036/wlserver/server/lib/weblogic_sp.jar:/middleware/wld-pro-04/wl1036/wlserver/server/lib/weblogic.jar:/middleware/wld-pro-04/wl1036/modules/features/weblogic.server.modules_10.3.6.0.jar:/middleware/wld-pro-04/wl1036/wlserver/server/lib/webservices.jar:/middleware/wld-pro-04/wl1036/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/middleware/wld-pro-04/wl1036/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:

Debug:

Debug: PATH=/middleware/wld-pro-04/wl1036/wlserver/server/bin:/middleware/wld-pro-04/wl1036/modules/org.apache.ant_1.7.1/bin:/usr/jdk/instances/jdk1.7.0/jre/bin:/usr/jdk/instances/jdk1.7.0/bin:/usr/bin:/usr/sbin

Debug:

Debug: Your environment has been set.

Debug:

Debug: Initializing WebLogic Scripting Tool (WLST) ...

Debug:

Debug: Welcome to WebLogic Server Administration Scripting Shell

Debug:

Debug: Type help() for help on available commands

Debug:

Debug: wls:/offline> wls:/offline> wls:/offline> wls:/offline> wls:/offline> wls:/offline> Connecting to t3://puppet-pil00.msc.es:10400 with userid weblogic ...

Debug: Successfully connected to Admin Server 'wls-pro-04-sa-adm-n0' that belongs to domain 'wld-pro-04'.

Debug:

Debug: Warning: An insecure protocol was used to connect to the

Debug: server. To ensure on-the-wire security, the SSL port or

Debug: Admin port should be used instead.

Debug:

Debug: Storing state
Debug: Stored state in 3.91 seconds
Error: Failed to apply catalog: timeout on reading expected output from daemon process.
Debug: Dynamically-bound server lookup failed, falling back to report_server setting
Debug: Dynamically-bound port lookup failed; falling back to report_port setting
Debug: Closing connection for https://vm-lab-linux-1.msc.es:8140
Debug: Creating new connection for https://vm-lab-linux-1.msc.es:8140
Debug: Starting connection for https://vm-lab-linux-1.msc.es:8140
Debug: Caching connection for https://vm-lab-linux-1.msc.es:8140
Debug: Closing connection for https://vm-lab-linux-1.msc.es:8140

Nodemanager and admin server are up & running properly: root@puppet-pil00:/domains/wld-pro-04/config# netstat -na |grep 10400 ::ffff:10.15.7.104.10400 . 0 0 128000 0 LISTEN root@puppet-pil00:/domains/wld-pro-04/config# netstat -na |grep 5104 ::ffff:10.15.7.104.5104 . 0 0 128000 0 LISTEN

I need assistance to trace this error.

Thanks in advance,

Raúl

biemond commented 7 years ago

can you do it for wls_machine first.
security is always tricky , did you change something which requires a restart of the adminserver.

rssh22 commented 7 years ago

Hi,

I didn't make any change on the domain. I just create the domain from scratch.

I've tried wls_machine first with a similar error:

Debug: Prefetching simple resources for wls_machine
Debug: index wls_machine
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Debug: Dynamically-bound server lookup failed, falling back to server setting
Debug: Dynamically-bound port lookup failed; falling back to masterport setting
Debug: Closing connection for https://vm-lab-linux-1.msc.es:8140
Debug: Creating new connection for https://vm-lab-linux-1.msc.es:8140
Debug: Starting connection for https://vm-lab-linux-1.msc.es:8140
Debug: Caching connection for https://vm-lab-linux-1.msc.es:8140
Debug: Executing: wlstScript with action index
Debug: domain found wld-pro-04
Prepare to run: /tmp/wlstScript20170309-23570-1qpfem4.py,bea,wld-pro-04,/middleware/wld-pro-04/wl1036/wlserver,weblogic,welcome1,t3://puppet-pil00.msc.es:10400
vvv===================================================================

m = ls('/Machines',returnMap='true')

f = open_file("name;machinetype;nmtype;listenaddress;listenport;domain")
for token in m:
    print '___'+token+'___'
    cd('/Machines/'+token)
    type = get('Type')
    cd('NodeManager/'+token)

    listenAddress = get('ListenAddress')
    if not (listenAddress):
       listenAddress = ''

    add_index_entry(f, [domain+'/'+token,type,get('NMType'),listenAddress,str(get('ListenPort')),domain])

f.close()
report_back_success()
^^^====================================================================
Debug: no custom trust
Info: Starting the wls daemon for domain wld-pro-04
Debug: Defining common methods...
Debug: Dynamically-bound server lookup failed, falling back to server setting
Debug: Dynamically-bound port lookup failed; falling back to masterport setting
Debug: Using cached connection for https://vm-lab-linux-1.msc.es:8140
Debug: Caching connection for https://vm-lab-linux-1.msc.es:8140
Info: Executing wls-script /tmp/wlstCommonScript.py20170309-23570-5tkr6d
Info: Executing wls-script /tmp/wlstScript20170309-23570-1qpfem4.py with timeout = 120
Debug: Passing domain wld-pro-04
Debug: Passing use_default_value_when_empty False
Debug: Passing credentials to WLST
Info: Connecting to wls on url t3://puppet-pil00.msc.es:10400
Debug: Oracle Corporation SunOS 5.11      11.2    June 2014 [00]

Debug: CLASSPATH=/middleware/wld-pro-04/wl1036/patch_wls1036/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/middleware/wld-pro-04/wl1036/patch_ocp371/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/usr/jdk/instances/jdk1.7.0/lib/tools.jar:/middleware/wld-pro-04/wl1036/wlserver/server/lib/weblogic_sp.jar:/middleware/wld-pro-04/wl1036/wlserver/server/lib/weblogic.jar:/middleware/wld-pro-04/wl1036/modules/features/weblogic.server.modules_10.3.6.0.jar:/middleware/wld-pro-04/wl1036/wlserver/server/lib/webservices.jar:/middleware/wld-pro-04/wl1036/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/middleware/wld-pro-04/wl1036/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:

Debug:

Debug: PATH=/middleware/wld-pro-04/wl1036/wlserver/server/bin:/middleware/wld-pro-04/wl1036/modules/org.apache.ant_1.7.1/bin:/usr/jdk/instances/jdk1.7.0/jre/bin:/usr/jdk/instances/jdk1.7.0/bin:/usr/bin:/usr/sbin

Debug:

Debug: Your environment has been set.

Debug:

Debug: Initializing WebLogic Scripting Tool (WLST) ...

Debug:

Debug: Welcome to WebLogic Server Administration Scripting Shell

Debug:

Debug: Type help() for help on available commands

Debug:

Debug: wls:/offline> wls:/offline> wls:/offline> wls:/offline> wls:/offline> wls:/offline> Connecting to t3://puppet-pil00.msc.es:10400 with userid weblogic ...

Debug: Successfully connected to Admin Server 'wls-pro-04-sa-adm-n0' that belongs to domain 'wld-pro-04'.

Debug:

Debug: Warning: An insecure protocol was used to connect to the

Debug: server. To ensure on-the-wire security, the SSL port or

Debug: Admin port should be used instead.

Debug:

Debug: wls:/wld-pro-04/serverConfig> dr--   puppet-pil00

Debug:

Debug: Storing state
Debug: Stored state in 3.87 seconds
Error: Failed to apply catalog: timeout on reading expected output from daemon process.
Debug: Dynamically-bound server lookup failed, falling back to report_server setting
Debug: Dynamically-bound port lookup failed; falling back to report_port setting
Debug: Closing connection for https://vm-lab-linux-1.msc.es:8140
Debug: Creating new connection for https://vm-lab-linux-1.msc.es:8140
Debug: Starting connection for https://vm-lab-linux-1.msc.es:8140
Debug: Caching connection for https://vm-lab-linux-1.msc.es:8140
Debug: Closing connection for https://vm-lab-linux-1.msc.es:8140
biemond commented 7 years ago

ok but this helps

so it can be that it cannot write this output file .

what you can do is adding debug statements to wls_index of wls_machines https://github.com/biemond/biemond-orawls/blob/master/files/providers/wls_machine/index.py.erb

or run this yourself in wlst.sh

m = ls('/Machines',returnMap='true')

f = open_file("name;machinetype;nmtype;listenaddress;listenport;domain")
for token in m:
    print '___'+token+'___'
    cd('/Machines/'+token)
    type = get('Type')
    cd('NodeManager/'+token)

    listenAddress = get('ListenAddress')
    if not (listenAddress):
       listenAddress = ''

    add_index_entry(f, [domain+'/'+token,type,get('NMType'),listenAddress,str(get('ListenPort')),domain])

f.close()
rssh22 commented 7 years ago

Hi Biemond,

YES!!! I haven't set the log_dir parameter of the orawls::domain class.

Thanks again!!!

Raúl