Exception when getting data from SAP HANA

Hi! I am catching a bug with RowBinary format insert requests via jdbc-bridge when getting data from SAP HANA For very small tables, this error is not observed

clickhouse-server:22.1.3.7 jdbc-bridge:2.1

Request

insert into buf_sapabap."/AGRI/GLFLATV" (MANDT, TPLNR_FL, CLINT, ATINN, ATZHL, CLASS, ATWRT, ATFLV, ATAWE, ATFLB, ATAW1, ATCOD, ATTLV, ATTLB, ATPRZ, ATINC, ATAUT, DATUV, ATIMB, ATZIS, ATSRT, ATVGLART, DELETED, ERROR, ERNAM, ERDAT, ERZET, AENAM, AEDAT, AEZET) select MANDT, TPLNR_FL, CLINT, ATINN, ATZHL, CLASS, ATWRT, ATFLV, ATAWE, ATFLB, ATAW1, ATCOD, ATTLV, ATTLB, ATPRZ, ATINC, ATAUT, DATUV, ATIMB, ATZIS, ATSRT, ATVGLART, DELETED, ERROR, ERNAM, ERDAT, ERZET, AENAM, AEDAT, AEZET from jdbc('hanaHAP', 'select * from "/AGRI/GLFLATV"');

Log

DB::Exception: Cannot read from istream at offset 5242880: While executing BinaryRowInputFormat: While executing JDBC. Stack trace:

0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xa82d07a in /usr/bin/clickhouse
1. DB::ReadBufferFromIStream::nextImpl() @ 0x1158ade0 in /usr/bin/clickhouse
2. DB::detail::ReadWriteBufferFromHTTPBase<std::__1::shared_ptr<DB::UpdatableSession> >::nextImpl() @ 0x1157ce79 in /usr/bin/clickhouse
3. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0xa86b2d9 in /usr/bin/clickhouse
4. DB::SerializationString::deserializeBinary(DB::IColumn&, DB::ReadBuffer&) const @ 0x1328b7bb in /usr/bin/clickhouse
5. DB::BinaryFormatReader::readField(DB::IColumn&, std::__1::shared_ptr<DB::IDataType const> const&, std::__1::shared_ptr<DB::ISerialization const> const&, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) @ 0x148b8b11 in /usr/bin/clickhouse
6. DB::RowInputFormatWithNamesAndTypes::readRow(std::__1::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, DB::RowReadExtension&) @ 0x148bbac5 in /usr/bin/clickhouse
7. DB::IRowInputFormat::generate() @ 0x148add10 in /usr/bin/clickhouse
8. DB::ISource::tryGenerate() @ 0x148414b5 in /usr/bin/clickhouse
9. DB::ISource::work() @ 0x1484107a in /usr/bin/clickhouse
10. DB::ExecutionThreadContext::executeTask() @ 0x14860b23 in /usr/bin/clickhouse
11. DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*) @ 0x1485539e in /usr/bin/clickhouse
12. DB::PipelineExecutor::executeStep(std::__1::atomic<bool>*) @ 0x14854aa0 in /usr/bin/clickhouse
13. DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x148652ec in /usr/bin/clickhouse
14. ? @ 0x140bb304 in /usr/bin/clickhouse
15. DB::ISource::tryGenerate() @ 0x148414b5 in /usr/bin/clickhouse
16. DB::ISource::work() @ 0x1484107a in /usr/bin/clickhouse
17. DB::SourceWithProgress::work() @ 0x14a8c662 in /usr/bin/clickhouse
18. DB::ExecutionThreadContext::executeTask() @ 0x14860b23 in /usr/bin/clickhouse
19. DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*) @ 0x1485539e in /usr/bin/clickhouse
20. DB::PipelineExecutor::executeImpl(unsigned long) @ 0x148541e9 in /usr/bin/clickhouse
21. DB::PipelineExecutor::execute(unsigned long) @ 0x14853ef8 in /usr/bin/clickhouse
22. ? @ 0x148532a8 in /usr/bin/clickhouse
23. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xa86f4b7 in /usr/bin/clickhouse
24. ? @ 0xa872ebd in /usr/bin/clickhouse
25. ? @ 0x7f7f6f259609 in ?
26. __clone @ 0x7f7f6f180293 in ?

Connection to the bridge is lost after an error (probably restart service)

DB::Exception: clickhouse-jdbc-bridge is not running. Please, start it manually. Stack trace:

DB::Exception: Received error from remote server /columns_info?connection_string=hanaHAP&table=select%20%2A%20from%20AUFK&external_table_functions_use_nulls=true. HTTP status code: 500 Internal Server Error, body: NamedDataSource [hanaHAP] does not exist!. Stack trace:

DataSource settings

{
  "hanaHAP": {
    "driverUrls": [
      "https://repo1.maven.org/maven2/com/sap/cloud/db/jdbc/ngdbc/2.14.9/ngdbc-2.14.9.jar"
    ],
    "driverClassName": "com.sap.db.jdbc.Driver",
    "jdbcUrl": "jdbc:sap://<host>:<port>/?closeHandlesOnFinalize=TRUE&prefetch=FALSE",
    "connectionTestQuery": "SELECT 1 FROM SYS.DUMMY",
    "username": "",
    "password": "!",
    "schema":"SAPABAP1",
    "initializationFailTimeout": 0,
    "minimumIdle": 0,
    "maximumPoolSize": 10,
    "connectionTimeout": 30000,
    "readOnly": true
  }
}

HANA Table with 111 321 rows

@EndUserText.label : 'Terrain Attribute Values'
@AbapCatalog.enhancementCategory : #NOT_CLASSIFIED
@AbapCatalog.tableCategory : #TRANSPARENT
@AbapCatalog.deliveryClass : #A
@AbapCatalog.dataMaintenance : #LIMITED
define table /agri/glflatv {
  key mandt    : mandt not null;
  key tplnr_fl : /agri/gltplnr_fl not null;
  key clint    : clint not null;
  key atinn    : atinn not null;
  key atzhl    : atzhl not null;
  class        : /agri/gatgrp;
  atwrt        : /agri/gatwrt;
  atflv        : atflv;
  atawe        : msehi;
  atflb        : atflb;
  ataw1        : msehi;
  atcod        : atcod;
  attlv        : attlv;
  attlb        : attlb;
  atprz        : atprz;
  atinc        : atinc;
  ataut        : /agri/gataut;
  datuv        : datuv not null;
  include auspaddf;
  deleted      : del not null;
  error        : co_error not null;
  include /agri/s_gadmdata;

}

Clickhouse machine cpu and memory usage

ClickHouse / clickhouse-jdbc-bridge

Exception when getting data from SAP HANA #164