apache / druid

Apache Druid: a high performance real-time analytics database.
https://druid.apache.org/
Apache License 2.0
13.51k stars 3.71k forks source link

Druid Tasks failing with IllegalStateException in 29.0.1 version #16435

Open laxmikantbpandhare opened 6 months ago

laxmikantbpandhare commented 6 months ago

Please provide a detailed title (e.g. "Broker crashes when using TopN query with Bound filter" instead of just "Broker crashes").

Affected Version

29.0.1

Description

Please include as much detailed information about the problem as possible. Tasks were running in the 27.0.0 version and I upgraded it to 29.0.1 and it started failing.

The exact Error is: 1 error at [Source: (File) line:1, column, 1085] (through reference chain: org.apache.druid.indexing.kafka.KafkaIndexTask["dataSchema"] -> org.apache.druid.segment.indexing.DataSchema["transformSpec"]

while locating org.apache.druid.server.metrics.DataSourceTaskIdHolder 1) Problem parsing object at prefic[druid.lookup]: Cannot construct instance of 'org.apache.druid.query.LookupListeningAnnouncerConfig'. problem: unable to provision, see the following errors: 1) Error in custom provider, java.lang.IllegalStateException: This is a proxy used to support circular reference. The object we're proxying is not constructed yet. Please wait until after injection has completed to use this object.

I got these error messages from middlemanager logs.

laxmikantbpandhare commented 6 months ago

Same error happened with one more job

1 error at [Source: (File) line:1, column, 2535] (through reference chain: org.apache.druid.msq.indexing.MSQCOntrollerTask["spec"] -> org.apache.druid.query.scan.ScanQuery["virtualColumns"]

gianm commented 6 months ago

Thank you for the report.

I wonder if you have the ServiceStatusMonitor enabled? There is an issue with it which will be fixed in the next release: https://github.com/apache/druid/pull/16140

laxmikantbpandhare commented 6 months ago

Exactly @gianm. ServiceStatusMonitor is enabled in our Druid services. For time being, I reverted it back to 27.0.0

May I know in which Druid release we can expect this change 29.0.2?

acherla commented 6 months ago

Will be merged in 30.0

laxmikantbpandhare commented 6 months ago

Great, thank you for your reply.

On Wed, May 15, 2024 at 12:44 PM Arun Cherla @.***> wrote:

Will be merged in 30.0

— Reply to this email directly, view it on GitHub https://github.com/apache/druid/issues/16435#issuecomment-2113332511, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALHC3KF44KGE42XJC3MRDIDZCO3IPAVCNFSM6AAAAABHRLT57OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJTGMZTENJRGE . You are receiving this because you authored the thread.Message ID: @.***>