search

ibmbpm / bpm_analytics

Technology Demonstration for the IBM BPM Analytics solution based on Elasticsearch and Kibana
15 stars 7 forks source link

Release v8.6.0.201712 EventSummaryAgent failed with exception: com.ibm.bpm.mon.oi.facade.EventFacadeException #5

Open zhangs-ibm opened 6 years ago

zhangs-ibm commented 6 years ago

If your process include service flow, the EventSummaryAgent will fail with the following exception:

[Dec 20, 2017 8:38:21 AM ] 00000020 WARNING: com.ibm.bpm.mon.oi.facade.ESEventSourceFacadeImpl.getProcessEventsByShortProcessInstanceId - CWMCD0016W: An error occurred while the progress data was loaded. Root cause: org.elasticsearch.client.ResponseException: POST http://localhost:9200/bpm-events/ProcessEvent/_search: HTTP/1.1 400 Bad Request
{"error":{"root_cause":[{"type":"query_shard_exception","reason":"failed to create query: {\n  \"bool\" : {\n    \"must\" : [\n      {\n        \"match_phrase\" : {\n          \"bpmSystemId\" : {\n            \"query\" : \"84b7bbfb-26fc-44ba-a8d7-f39740e2bb55\",\n            \"slop\" : 0,\n            \"boost\" : 1.0\n          }\n        }\n      },\n      {\n        \"match_phrase\" : {\n          \"processInstanceId\" : {\n            \"query\" : \"19f77e5b-30f7-4b4f-84b9-13eb343026a6\",\n            \"slop\" : 0,\n            \"boost\" : 1.0\n          }\n        }\n      }\n    ],\n    \"disable_coord\" : false,\n    \"adjust_pure_negative\" : true,\n    \"boost\" : 1.0\n  }\n}","index_uuid":"MzLRUWDhRIqyYCZ5qc9qmQ","index":"bpm-events"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"bpm-events","node":"RbXU3AvkSQqbgPCxqnZUPg","reason":{"type":"query_shard_exception","reason":"failed to create query: {\n  \"bool\" : {\n    \"must\" : [\n      {\n        \"match_phrase\" : {\n          \"bpmSystemId\" : {\n            \"query\" : \"84b7bbfb-26fc-44ba-a8d7-f39740e2bb55\",\n            \"slop\" : 0,\n            \"boost\" : 1.0\n          }\n        }\n      },\n      {\n        \"match_phrase\" : {\n          \"processInstanceId\" : {\n            \"query\" : \"19f77e5b-30f7-4b4f-84b9-13eb343026a6\",\n            \"slop\" : 0,\n            \"boost\" : 1.0\n          }\n        }\n      }\n    ],\n    \"disable_coord\" : false,\n    \"adjust_pure_negative\" : true,\n    \"boost\" : 1.0\n  }\n}","index_uuid":"MzLRUWDhRIqyYCZ5qc9qmQ","index":"bpm-events","caused_by":{"type":"number_format_exception","reason":"For input string: \"19f77e5b-30f7-4b4f-84b9-13eb343026a6\""}}}]},"status":400}
[Dec 20, 2017 8:38:21 AM ] 00000020 SEVERE: com.ibm.bpm.mon.oi.combine.MergeJob.execute - CWMCD2015E: The Event Summary Agent cannot process job 84b7bbfb-26fc-44ba-a8d7-f39740e2bb55_ProcessSummary_19f77e5b-30f7-4b4f-84b9-13eb343026a6.
com.ibm.bpm.mon.oi.facade.EventFacadeException: CWMCD0012E: An unexpected exception was thrown when the data was retrieved.
    at com.ibm.bpm.mon.oi.facade.ESEventSourceFacadeImpl.getProcessEventsByShortProcessInstanceId(ESEventSourceFacadeImpl.java:312)
    at com.ibm.bpm.mon.oi.aggregate.ProcessAndActivityAggregatorImpl.aggregatProcessEvent(ProcessAndActivityAggregatorImpl.java:178)
    at com.ibm.bpm.mon.oi.aggregate.ProcessAndActivityAggregatorImpl.doJob(ProcessAndActivityAggregatorImpl.java:155)
    at com.ibm.bpm.mon.oi.aggregate.ProcessAndActivityAggregatorImpl.doJob(ProcessAndActivityAggregatorImpl.java:36)
    at com.ibm.bpm.mon.oi.combine.MergeJob.execute(MergeJob.java:40)
    at com.ibm.bpm.mon.oi.concurrent.JobQueue$JobWrapper.run(JobQueue.java:323)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
[Dec 20, 2017 8:38:21 AM ] 00000020 WARNING: com.ibm.bpm.mon.oi.facade.ESEventSourceFacadeImpl.getProcessEventsByShortProcessInstanceId - CWMCD0016W: An error occurred while the progress data was loaded. Root cause: org.elasticsearch.client.ResponseException: POST http://localhost:9200/bpm-events/ProcessEvent/_search: HTTP/1.1 400 Bad Request
{"error":{"root_cause":[{"type":"query_shard_exception","reason":"failed to create query: {\n  \"bool\" : {\n    \"must\" : [\n      {\n        \"match_phrase\" : {\n          \"bpmSystemId\" : {\n            \"query\" : \"84b7bbfb-26fc-44ba-a8d7-f39740e2bb55\",\n            \"slop\" : 0,\n            \"boost\" : 1.0\n          }\n        }\n      },\n      {\n        \"match_phrase\" : {\n          \"processInstanceId\" : {\n            \"query\" : \"19f77e5b-30f7-4b4f-84b9-13eb343026a6\",\n            \"slop\" : 0,\n            \"boost\" : 1.0\n          }\n        }\n      }\n    ],\n    \"disable_coord\" : false,\n    \"adjust_pure_negative\" : true,\n    \"boost\" : 1.0\n  }\n}","index_uuid":"MzLRUWDhRIqyYCZ5qc9qmQ","index":"bpm-events"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"bpm-events","node":"RbXU3AvkSQqbgPCxqnZUPg","reason":{"type":"query_shard_exception","reason":"failed to create query: {\n  \"bool\" : {\n    \"must\" : [\n      {\n        \"match_phrase\" : {\n          \"bpmSystemId\" : {\n            \"query\" : \"84b7bbfb-26fc-44ba-a8d7-f39740e2bb55\",\n            \"slop\" : 0,\n            \"boost\" : 1.0\n          }\n        }\n      },\n      {\n        \"match_phrase\" : {\n          \"processInstanceId\" : {\n            \"query\" : \"19f77e5b-30f7-4b4f-84b9-13eb343026a6\",\n            \"slop\" : 0,\n            \"boost\" : 1.0\n          }\n        }\n      }\n    ],\n    \"disable_coord\" : false,\n    \"adjust_pure_negative\" : true,\n    \"boost\" : 1.0\n  }\n}","index_uuid":"MzLRUWDhRIqyYCZ5qc9qmQ","index":"bpm-events","caused_by":{"type":"number_format_exception","reason":"For input string: \"19f77e5b-30f7-4b4f-84b9-13eb343026a6\""}}}]},"status":400}
[Dec 20, 2017 8:38:21 AM ] 00000020 SEVERE: com.ibm.bpm.mon.oi.combine.MergeJob.execute - CWMCD2015E: The Event Summary Agent cannot process job 84b7bbfb-26fc-44ba-a8d7-f39740e2bb55_ProcessSummary_19f77e5b-30f7-4b4f-84b9-13eb343026a6.
com.ibm.bpm.mon.oi.facade.EventFacadeException: CWMCD0012E: An unexpected exception was thrown when the data was retrieved.
    at com.ibm.bpm.mon.oi.facade.ESEventSourceFacadeImpl.getProcessEventsByShortProcessInstanceId(ESEventSourceFacadeImpl.java:312)
    at com.ibm.bpm.mon.oi.aggregate.ProcessAndActivityAggregatorImpl.aggregatProcessEvent(ProcessAndActivityAggregatorImpl.java:178)
    at com.ibm.bpm.mon.oi.aggregate.ProcessAndActivityAggregatorImpl.doJob(ProcessAndActivityAggregatorImpl.java:155)
    at com.ibm.bpm.mon.oi.aggregate.ProcessAndActivityAggregatorImpl.doJob(ProcessAndActivityAggregatorImpl.java:36)
    at com.ibm.bpm.mon.oi.combine.MergeJob.execute(MergeJob.java:40)
    at com.ibm.bpm.mon.oi.concurrent.JobQueue$JobWrapper.run(JobQueue.java:323)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
zhangs-ibm commented 6 years ago

The BPM Analytics do not support the event type generated by the service flow, please update the subscriptions at the ${BPM_ROOT}/BPM/Lombardi/tools/def/SampleConfigureJSONEventsToJMS.py to filter out the service flow event. Change the subscription to:

subscriptions = [
 '*/*/BPD/*/*/*/*'
]

Then run: wsadmin.sh -lang jython -f EnableBPMAnalytics.py --update=subscriptions

Since the service flow events are emitted at the Elasticsearch, the data need to be cleared before move forward.

  1. stop the EventSummaryAgent
  2. stop the BPMEventEmitter application
  3. delete the indices already created at the Elasticsearch, the index name is the one at the BPMEventEmitter configuration file
  4. start the BPMEventEmitter application
  5. start the EventSummaryAgent
AleToledoCR commented 6 years ago

Hello:

I've tried the sugested solution and executed the script for the subscriptions update. It failed. Then I tried to recreate the configuration using the modified SampleConfigureJSONEventsToJMS.py file. The same exception was thrown.

I'm using ES version 5.5.3 and BPM 8.6.0 with the last commulative fixpack.

Would you please help us resolving the issue? I'm very interested in making this work for the most recent version of BPM.

zhangs-ibm commented 6 years ago

Hi, The same exception was thrown. is the exception as at the EventSummaryAgent or the exception when run the EnableBPMAnalytics.py --update?

Would you paste the content of the defconfig.xml at the cell level(BPM_ROOT\profiles\DMGR_NAME\config\cells\CELL_NAME) at here?

AleToledoCR commented 6 years ago

Just before submit the information you asked (btw thanks for the answer) I reviewed the defconfig.xml file. I noticed that there was a dupplicate ProducerFor_jmsListenerForJSON entry one with the wrong configuration and another one with the correct one.

Once I deleted that file, fully resync the node and recreated the configuration using the modified SampleConfigureJSONEventsToJMS.py file everthing works fine.

Thanks for the hint with the defconfig.xml file.

The one thing I couldn't make to work is the wsadmin.sh -lang jython -f EnableBPMAnalytics.py --update=subscriptions command. I had to recreate that configuration from the very beggining, deleting the different resources I had created the first time and use for a second time the Enable IBM BPM Analytics instrucctions in the pdf file you supplied.

zhangs-ibm commented 6 years ago

Glad to hear the issue is solved, when run the EnableBPMAnalytics at second time, there is an redundance entry created, this issue will be fixed at the V860.201803.

If you run the --update with exception, this ifix can help you: https://www-945.ibm.com/support/fixcentral/swg/selectFixes?product=ibm%2FWebSphere%2FIBM+Business+Process+Manager+Express&fixids=8.6.0.201712-WS-BPM-IFJR58916&source=SAR&function=fixId&parent=ibm/WebSphere

gabrigam commented 6 years ago

Hi,

I have the same exceptions but seems that my defconfig.xml does not contains duplicate.

Also I'm using ES version 5.5.3 and BPM 8.6.0 with the last commulative fixpack.

Please help me for the solutions.

Thanks

Best Regards.

this is my defconfig.xml

<?xml version="1.0" encoding="UTF-8"?>

gabrigam commented 6 years ago

Hi,

my defconfig.xml in attachement.

Thanks defconfig.zip

zhangs-ibm commented 6 years ago

@gabrigam , if you got the same exception as the description of this issue, it cased by the EventSummaryAgent can not process the new ServiceFlow type event. You need filter out this type of events at the DEF configuration:

Please update the subscriptions at the ${BPM_ROOT}/BPM/Lombardi/tools/def/SampleConfigureJSONEventsToJMS.py to filter out the service flow event. Change the subscription to:

subscriptions = [
 '*/*/BPD/*/*/*/*'
]

Then run: wsadmin.sh -lang jython -f EnableBPMAnalytics.py --update=subscriptions

BTW, you need this ifix applied before run the EnableBPMAnalytics.py --update: If you run the --update with exception, this ifix can help you: https://www-945.ibm.com/support/fixcentral/swg/selectFixes?product=ibm%2FWebSphere%2FIBM+Business+Process+Manager+Express&fixids=8.6.0.201712-WS-BPM-IFJR58916&source=SAR&function=fixId&parent=ibm/WebSphere

gabrigam commented 6 years ago

Hi zhangs,

Thanks for your Help.

Now all works fine.

Gabriele

gabrigam commented 6 years ago

HI zhangs,

Please I need your help again.

I receive some errors during the dashboard installation on Kibana.

In attachements some images related to the installation steps.

Thanks in Advance.

Best Regards

Gabriele images.zip

gabrigam commented 6 years ago

Hi zhangs

Resolved.

[Kibana 5.5.x and later version] Enter bpm-summary*

I remove * and works fine.

Thanks

Gabriele