search

servicecatalog / development

An Enterprise-ready Cloud Services Management Software
http://openservicecatalogmanager.org
Apache License 2.0
39 stars 27 forks source link

master-indexer-domain server.log filled with too many ERRORS #564

Closed ghost closed 7 years ago

ghost commented 7 years ago

[Version] 16.1 fix5+ 2017/02/06

[How to reproduce]

  1. install ctmg
  2. carry out normal operations
  3. check server.log of master-indexer

[Expected result] no ERRORs, almost no WARNINGS

[Observed result] Filled with too many ERRORS which fill up the log file very fast and are killing disk space.

[Details] see attached file in last comment

GoebelL commented 7 years ago

Hugh. Lot's of ConnectException's. The JMS configuration doesn't look healthy.

StavrevaS commented 7 years ago

@SylviaDehn, please check the following:

Maybe the installation of indexer domain configures the JMS port wrong.

ghost commented 7 years ago

You are correct, Suzanna. Indexer domain was not running when I copied the file. So I am changing the bug to ERRORs that are in system.log of master indexer. It is filling up like crazy. 8476 is in domain.xml.

ghost commented 7 years ago

server.txt

ghost commented 7 years ago

image

ghost commented 7 years ago

See log files above and how many are created in written per minute ....

ghost commented 7 years ago

The problem I have, my disk fills up, with master-indexer errors. If I stop the master-indexer, then the disk fills up with bes-domain connect errors .... so the situation is not acceptable

StavrevaS commented 7 years ago

@SylviaDehn, please check if /imq/instances/imqbroker/lock file exists when the domain is stopped. If yes, please delete it and start the domain again. Do this for indexer and bes domains. This can be a reason of the embedded imq broker start problem which is reported in the master indexer log.

Please test with jdk8u60, I see jdk8u121 in the logs. There are maybe other problems with the higher versions.

[SEVERE] [rardeployment.start_failed] [javax.enterprise.resource.resourceadapter.com.sun.enterprise.connectors] [tid: _ThreadID=19 _ThreadName=RunLevelControllerThread-1486629720178] [timeMillis: 1486629766810] [levelValue: 1000] [[ RAR6035 : Resource adapter start failed. javax.resource.spi.ResourceAdapterInternalException: java.security.PrivilegedActionException: javax.resource.spi.ResourceAdapterInternalException: MQJMSRA_RA4001: start:Aborting:Exception starting EMBEDDED broker=Broker failed to start at com.sun.enterprise.connectors.jms.system.ActiveJmsResourceAdapter.startResourceAdapter(ActiveJmsResourceAdapter.java:510) at com.sun.enterprise.connectors.ActiveOutboundResourceAdapter.init(ActiveOutboundResourceAdapter.java:130) at com.sun.enterprise.connectors.inbound.ActiveInboundResourceAdapterImpl.init(ActiveInboundResourceAdapterImpl.java:90) at com.sun.enterprise.connectors.ActiveRAFactory.instantiateActiveResourceAdapter(ActiveRAFactory.java:141) at com.sun.enterprise.connectors.ActiveRAFactory.createActiveResourceAdapter(ActiveRAFactory.java:108) at com.sun.enterprise.connectors.service.ResourceAdapterAdminServiceImpl.createActiveResourceAdapter(ResourceAdapterAdminServiceImpl.java:210) at com.sun.enterprise.connectors.service.ResourceAdapterAdminServiceImpl.createActiveResourceAdapter(ResourceAdapterAdminServiceImpl.java:346) at com.sun.enterprise.connectors.ConnectorRuntime.createActiveResourceAdapter(ConnectorRuntime.java:405) at com.sun.enterprise.connectors.service.ConnectorService.loadDeferredResourceAdapter(ConnectorService.java:184) at com.sun.enterprise.connectors.ConnectorRuntime.loadDeferredResourceAdapter(ConnectorRuntime.java:1331) at com.sun.enterprise.connectors.inbound.ConnectorMessageBeanClient.getActiveResourceAdapter(ConnectorMessageBeanClient.java:315) at com.sun.enterprise.connectors.inbound.ConnectorMessageBeanClient.setup(ConnectorMessageBeanClient.java:166) at org.glassfish.ejb.mdb.MessageBeanContainer.(MessageBeanContainer.java:252) at org.glassfish.ejb.mdb.MessageBeanContainerFactory.createContainer(MessageBeanContainerFactory.java:63) at org.glassfish.ejb.startup.EjbApplication.loadContainers(EjbApplication.java:221) at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:291) at org.glassfish.ejb.startup.EjbDeployer.load(EjbDeployer.java:99)

DawidCh commented 7 years ago

[Details of fix] Default port for jms queue for domains was changed in gl4. So we updated installation process and set up correct port. The connection troubles should not occur again.

ghost commented 7 years ago

[Version] 16.1 pre fix7 2016/02/27 Problem still occurs. New logfile is attached. server_log.txt

DawidCh commented 7 years ago

The logs are connected with the issue #568 This fix is for jms related problems.

ghost commented 7 years ago

The other bug is closed, my original problem report is still valid. Do you want me to open a new bug?

ghost commented 7 years ago

verified 17.2