search

apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.58k stars 3.25k forks source link

[Bug] Failure allocating buffer #14715

Open gaoyan1998 opened 1 year ago

gaoyan1998 commented 1 year ago

Search before asking

Version

1.1

What's Wrong?

use flink sql read doris failed ,

Caused by: org.apache.doris.flink.exception.DorisRuntimeException: Failure allocating buffer. at org.apache.doris.flink.serialization.RowBatch.readArrow(RowBatch.java:128) ~[flink-doris-connector-1.15-1.2.0.jar:1.2.0] at org.apache.doris.flink.source.reader.DorisValueReader.hasNext(DorisValueReader.java:202) ~[flink-doris-connector-1.15-1.2.0.jar:1.2.0] at org.apache.doris.flink.source.split.DorisSplitRecords.nextRecordFromSplit(DorisSplitRecords.java:70) ~[flink-doris-connector-1.15-1.2.0.jar:1.2.0] at org.apache.doris.flink.source.split.DorisSplitRecords.nextRecordFromSplit(DorisSplitRecords.java:32) ~[flink-doris-connector-1.15-1.2.0.jar:1.2.0] at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:139) ~[flink-connector-base-1.15.2.jar:1.15.2] at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:385) ~[flink-streaming-java-1.15.2.jar:1.15.2] at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) ~[flink-streaming-java-1.15.2.jar:1.15.2] at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) ~[flink-streaming-java-1.15.2.jar:1.15.2] at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:519) ~[flink-streaming-java-1.15.2.jar:1.15.2] at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203) ~[flink-streaming-java-1.15.2.jar:1.15.2] at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:804) ~[flink-streaming-java-1.15.2.jar:1.15.2] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:753) ~[flink-streaming-java-1.15.2.jar:1.15.2] at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:948) ~[flink-runtime-1.15.2.jar:1.15.2] at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927) ~[flink-runtime-1.15.2.jar:1.15.2] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741) ~[flink-runtime-1.15.2.jar:1.15.2] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) ~[flink-runtime-1.15.2.jar:1.15.2] at java.lang.Thread.run(Unknown Source) ~[?:?]

What You Expected?

read data batch

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

Code of Conduct

DongLiang-0 commented 1 year ago

Which version of doris-flink-connector

gaoyan1998 commented 1 year ago

Contributor

1.2.0

gaoyan1998 commented 1 year ago

Contributor

1.2.0 flink version 1.15.2

DongLiang-0 commented 1 year ago

The reason for this is you jvm -XX:MaxDirectMemorySize not enough, you can increase -Xmx to solve this problem. reference: -XX:MaxDirectMemorySize

Try again after adjusted to see if it still happens

gaoyan1998 commented 1 year ago

The reason for this is you jvm -XX:MaxDirectMemorySize not enough, you can increase -Xmx to solve this problem. reference: -XX:MaxDirectMemorySize

Try again after adjusted to see if it still happens

both of be and fe ?

gaoyan1998 commented 1 year ago

fe

thx.

DongLiang-0 commented 1 year ago

The reason for this is you jvm -XX:MaxDirectMemorySize not enough, you can increase -Xmx to solve this problem. reference: -XX:MaxDirectMemorySize Try again after adjusted to see if it still happens

both of be and fe ?

try both