SAP / kafka-connect-sap

Kafka Connect SAP is a set of connectors, using the Apache Kafka Connect framework for reliably connecting Kafka with SAP systems
Apache License 2.0
122 stars 56 forks source link

Missing Table in Cloud Hana #113

Closed AndreasKlein closed 2 years ago

AndreasKlein commented 2 years ago

Hi @elakito

we have provisioned a Cloud Hana in a CF Space on BTP and are attempting to use the connector with it. Unfortunately the DB does not have a SYS.M_CS_PARTITIONS view for the partition's information.

Is that a different version of Hana, not supported yet? Is there a different table to be used? If so, would you accept a PR adjusting the behaviour?

We'd be grateful, if you'd shed some light on this.

Cheers

Andreas

elakito commented 2 years ago

@AndreasKlein Yes, SYS.M_CS_PARTITIONS was deprecated in 2.0 SP 05 but I didn't know it was already removed in the current version. I think the new tables mentioned in the documentation, TABLE_PARTITIONS and M_CS_TABLES are not a drop-in replacement table for M_CS_PARTITIONS and a few things would need to be adjusted for the switch. If you have time to look into it, that would be greatly appreciated. And a PR is welcome. Thanks, aki

elakito commented 2 years ago

@AndreasKlein It looks like the code base for Cloud Hana was split from Hana around SPS 05 and those deprecated tables in SPS 05 are not included in Cloud Hana version. According to Cloud Hana documentation of M_CS_PARTITIONS [1], one can replace M_CS_PARTITIONS with TABLE_PARTITIONS.

[1] Cloud HANA - M_CS_PARTITIONS (the table not available) https://help.sap.com/viewer/3c53bc7b58934a9795b6dd8c7e28cf05/hanacloud/en-US/fc102975eb1b4d4a83f0dfb890aaf896.html

[2] Clound HANA TABLE_PARTITIONS https://help.sap.com/viewer/c1d3f60099654ecfb3fe36ac93c121bb/2021_4_QRC/en-US/c81d9beea383495c99ad48789b983708.html

[3] HANA SPS 04 M_CS_PARTIONS (in SPS 05, the table is deprecated but still available) https://help.sap.com/viewer/4fe29514fd584807ac9f2a04f6754767/2.0.04/en-US/20ad3d1075191014bce8aba8320b5b94.html

[4] HANA SPS 05 TABLE_PARTITIONS https://help.sap.com/viewer/4fe29514fd584807ac9f2a04f6754767/2.0.05/en-US/c81d9beea383495c99ad48789b983708.html

AndreasKlein commented 2 years ago

@elakito I was just about to write you. From looking at it, M_CS_TABLES seems to be the one. TABLE_PARTITIONS is empty in our case and all necessary info seems present M_CS_TABLES: SCHEMA_NAME, TABLE_NAME, PART_ID, the latter being the partition number.

AndreasKlein commented 2 years ago

However, "empty" may be ok, bc it just means no partition. Happy to use TABLE_PARTITIONS

AndreasKlein commented 2 years ago

https://github.com/SAP/kafka-connect-sap/pull/114

elakito commented 2 years ago

@AndreasKlein Thanks. You are right. TABLE_PARTIONS doesn't seem to include non-partitioned tables even though its documentation actually indicates PART_ID = 0 is returned for non-partitioned tables. In contrast, M_CS_TABLES works for partitioned and non-partitioned tables. Thanks for the PR.