search

open-metadata / OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.
https://open-metadata.org
Apache License 2.0
5.66k stars 1.06k forks source link

changeEvent parsing failing #13251

Closed harshach closed 10 months ago

harshach commented 1 year ago

ERROR [2023-09-18 20:33:31,355] [ForkJoinPool-1-worker-17] o.o.s.e.ChangeEventHandler - Failed to capture the change event for method PUT due to java.lang.IllegalArgumentException: Logical Test Cases Added to Test Suite at org.openmetadata.schema.type.EventType.fromValue(EventType.java:51) at org.openmetadata.service.formatter.util.FormatterUtil.getChangeEventFromResponseContext(FormatterUtil.java:265) at org.openmetadata.service.events.ChangeEventHandler.process(ChangeEventHandler.java:65) at org.openmetadata.service.events.EventFilter.lambda$filter$1(EventFilter.java:78) at org.openmetadata.service.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:74) at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700) at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1692) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

chirag-madlani commented 1 year ago

Pasting some more ES failures here:


2023-09-21 14:45:21 172.16.239.1 - - [21/Sep/2023:09:15:21 +0000] "GET /api/v1/search/query?q=*raw_product_catalog*&from=0&size=25&index=table_search_index HTTP/1.1" 200 1456 "http://localhost:8585/table/sample_data.ecommerce_db.shopify.%22dim.api%2Fclient%22/table_queries?tableId=12df353d-874a-48a0-a119-266a5e0d7b78&query=55fc9d24-f9ad-4078-99b2-f83e7e5e61f6" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" 33
2023-09-21 14:45:21 ERROR [2023-09-21 09:15:21,053] [ForkJoinPool-1-worker-17] o.o.s.e.ChangeEventHandler - Failed to capture the change event for method PATCH due to 
2023-09-21 14:45:21 java.lang.NullPointerException: null
2023-09-21 14:45:21     at org.openmetadata.service.formatter.decorators.FeedMessageDecorator.getEntityUrl(FeedMessageDecorator.java:53)
2023-09-21 14:45:21     at org.openmetadata.service.formatter.entity.QueryFormatter.getFieldValue(QueryFormatter.java:56)
2023-09-21 14:45:21     at org.openmetadata.service.formatter.entity.QueryFormatter.transformQueryUsedIn(QueryFormatter.java:72)
2023-09-21 14:45:21     at org.openmetadata.service.formatter.entity.QueryFormatter.format(QueryFormatter.java:38)
2023-09-21 14:45:21     at org.openmetadata.service.formatter.util.FormatterUtil.getFormattedMessagesForAllFieldChange(FormatterUtil.java:240)
2023-09-21 14:45:21     at org.openmetadata.service.formatter.util.FormatterUtil.getFormattedMessages(FormatterUtil.java:194)
2023-09-21 14:45:21     at org.openmetadata.service.util.FeedUtils.getThreads(FeedUtils.java:80)
2023-09-21 14:45:21     at org.openmetadata.service.util.FeedUtils.getThreads(FeedUtils.java:53)
2023-09-21 14:45:21     at org.openmetadata.service.events.ChangeEventHandler.process(ChangeEventHandler.java:88)
2023-09-21 14:45:21     at org.openmetadata.service.events.EventFilter.lambda$filter$1(EventFilter.java:78)
2023-09-21 14:45:21     at org.openmetadata.service.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:74)
2023-09-21 14:45:21     at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
2023-09-21 14:45:21     at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1692)
2023-09-21 14:45:21     at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
2023-09-21 14:45:21     at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
2023-09-21 14:45:21     at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
2023-09-21 14:45:21     at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
2023-09-21 14:45:21     at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
2023-09-21 14:45:21 INFO [2023-09-21 09:15:21,100] [dw-2183] o.o.s.s.WebSocketManager - Client from: 450cc995-7b53-4768-8327-81232cd0efab with Remote Address:[172.16.239.1] disconnected.
2023-09-21 14:45:21 INFO [2023-09-21 09:15:21,100] [dw-2182] o.o.s.s.WebSocketManager - Client from: 450cc995-7b53-4768-8327-81232cd0efab with Remote Address:[172.16.239.1] disconnected.
chirag-madlani commented 1 year ago 
2023-09-21 02:27:07 INFO [2023-09-20 20:57:07,895] [ForkJoinPool-1-worker-61] o.o.s.e.AuditEventHandler - Added audit log entry: org.openmetadata.schema.type.AuditLog@493ce413[method=PUT,responseCode=200,path=v1/dataQuality/testCases/logicalTestCases,userName=admin,entityId=431d5d2d-d8e3-4e28-a7e5-695d9ea196a4,entityType=testSuite,timestamp=1695243427895]
2023-09-21 02:27:07 ERROR [2023-09-20 20:57:07,895] [ForkJoinPool-1-worker-47] o.o.s.e.ChangeEventHandler - Failed to capture the change event for method PUT due to 
2023-09-21 02:27:07 java.lang.IllegalArgumentException: Logical Test Cases Added to Test Suite
2023-09-21 02:27:07     at org.openmetadata.schema.type.EventType.fromValue(EventType.java:51)
2023-09-21 02:27:07     at org.openmetadata.service.formatter.util.FormatterUtil.getChangeEventFromResponseContext(FormatterUtil.java:265)
2023-09-21 02:27:07     at org.openmetadata.service.events.ChangeEventHandler.process(ChangeEventHandler.java:65)
2023-09-21 02:27:07     at org.openmetadata.service.events.EventFilter.lambda$filter$1(EventFilter.java:78)
2023-09-21 02:27:07     at org.openmetadata.service.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:74)
2023-09-21 02:27:07     at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
2023-09-21 02:27:07     at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1692)
2023-09-21 02:27:07     at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
2023-09-21 02:27:07     at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
2023-09-21 02:27:07     at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
2023-09-21 02:27:07     at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
2023-09-21 02:27:07     at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
2023-09-21 02:27:07 172.16.239.5 - - [20/Sep/2023:20:57:07 +0000] "PUT /api/v1/dataQuality/testCases/logicalTestCases HTTP/1.1" 200 1869 "-" "python-requests/2.31.0" 14
2023-09-21 02:27:07 127.0.0.1 - - [20/Sep/2023:20:57:07 +0000] "GET /api/v1/system/config/jwks HTTP/1.1" 200 454 "-" "Java/11.0.20" 0
2023-09-21 02:27:07 WARN [2023-09-20 20:57:07,919] [I/O dispatcher 142] o.e.c.RestClient - request [POST http://elasticsearch:9200/test_case_search_index/_update/80919f87-280a-42c1-9f77-5e68a47d31f7?refresh=true&timeout=1m] returned 1 warnings: [299 Elasticsearch-7.16.3-4e6e4eab2297e949ec994e688dad46290d018022 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."]
harshach commented 1 year ago

@chirag-madlani none of these will impact search or any other functionality except for alerts or feed publishing

harshach commented 12 months ago

@mohityadav766 lets take care of this as part of notifications work you are doing

mohityadav766 commented 10 months ago

this is fixed here