search

apache / druid

Apache Druid: a high performance real-time analytics database.
https://druid.apache.org/
Apache License 2.0
13.49k stars 3.7k forks source link

org.apache.druid.java.util.common.ISE: Could not allocate segment for row with timestamp #15297

Open Sixty-one opened 1 year ago

Sixty-one commented 1 year ago

Please provide a detailed title (e.g. "Broker crashes when using TopN query with Bound filter" instead of just "Broker crashes").

Affected Version

0.22.1 The Druid version where the problem was encountered. 0.22.1

Description

The steps are as follows： 1.Durid is running normally，Write data through kafka 2.Execute command to modify operating system time，date -s "2023-10-04 15:30:00" 3.restart druid service

Please include as much detailed information about the problem as possible.

errros infos: 2023-11-01T02:25:56,503 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Encountered exception while running task. org.apache.druid.java.util.common.ISE: Could not allocate segment for row with timestamp[2023-11-04T07:30:09.000Z] at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.runInternal(SeekableStreamIndexTaskRunner.java:665) ~[druid-indexing-service-0.22.1.jar:0.22.1] at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.run(SeekableStreamIndexTaskRunner.java:263) [druid-indexing-service-0.22.1.jar:0.22.1] at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTask.run(SeekableStreamIndexTask.java:146) [druid-indexing-service-0.22.1.jar:0.22.1] at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:471) [druid-indexing-service-0.22.1.jar:0.22.1] at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:443) [druid-indexing-service-0.22.1.jar:0.22.1] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_322] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_322] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_322] at java.lang.Thread.run(Thread.java:750) [?:1.8.0_322] 2023-11-01T02:25:56,530 INFO [task-runner-0-priority-0] org.apache.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: { "id" : "index_kafka_cache_flow_section_23e8131b930cc09_mnhgkclm", "status" : "FAILED", "duration" : 5036, "errorMsg" : "org.apache.druid.java.util.common.ISE: Could not allocate segment for row with timestamp[2023-11-04T...", "location" : { "host" : null, "port" : -1, "tlsPort" : -1 } }

so how to solve the problem？

acherla commented 9 months ago

@abhagraw Bumping this one up as we still see this error even in 27.0.0, im wondering if this has to do with time clock synchronization?

2024-01-19T02:44:18,169 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner - Encountered exception while running task.
org.apache.druid.java.util.common.ISE: Could not allocate segment for row with timestamp[2024-01-18T14:59:54.450Z]
    at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.runInternal(SeekableStreamIndexTaskRunner.java:680) ~[druid-indexing-service-27.0.0.jar:27.0.0]
    at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTaskRunner.run(SeekableStreamIndexTaskRunner.java:266) ~[druid-indexing-service-27.0.0.jar:27.0.0]
    at org.apache.druid.indexing.seekablestream.SeekableStreamIndexTask.runTask(SeekableStreamIndexTask.java:151) ~[druid-indexing-service-27.0.0.jar:27.0.0]
    at org.apache.druid.indexing.common.task.AbstractTask.run(AbstractTask.java:173) ~[druid-indexing-service-27.0.0.jar:27.0.0]
    at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:477) ~[druid-indexing-service-27.0.0.jar:27.0.0]
    at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:449) ~[druid-indexing-service-27.0.0.jar:27.0.0]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_392]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_392]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_392]
    at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_392
github-actions[bot] commented 1 week ago

This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions.