SAP / kafka-connect-sap

Kafka Connect SAP is a set of connectors, using the Apache Kafka Connect framework for reliably connecting Kafka with SAP systems
Apache License 2.0
119 stars 54 forks source link

A normal numerical value of a SAP hana table is shown as scientific notation,when it is consumed. #108

Closed lawrenceee1 closed 2 years ago

lawrenceee1 commented 2 years ago

This is the hana connector config: { "connector.class": "com.sap.kafka.connect.source.hana.HANASourceConnector", "connection.password": "soFas2048H", "transforms.cast.type": "org.apache.kafka.connect.transforms.Cast$Value", "transforms.cast.spec": "ZWERT:float64,ZMENG:float64,UMZIZ:float64,UMZIN:float64,SMENG:float64,ABLFZ:float64,ABSFZ:float64,KBVER:float64,KEVER:float64,UEBTO:float64,UNTTO:float64,NETWR:float64,ANTLF:float64,KWMENG:float64,LSMENG:float64,KBMENG:float64,KLMENG:float64,UMVKZ:float64,UMVKN:float64,BRGEW:float64,NTGEW:float64,VOLUM:float64,STPOS:float64,VBEAF:float64,VBEAV:float64,NETPR:float64,KPEIN:float64,LFMNG:float64,WAVWR:float64,KZWI1:float64,KZWI2:float64,KZWI3:float64,KZWI4:float64,KZWI5:float64,KZWI6:float64,STCUR:float64,KMPMG:float64,CMPRE:float64,CMKUA:float64,MWSBP:float64,CMTD_DELIV_QTY_SU:float64,REQQTY_BU:float64,IFRS15_TOTAL_SSP:float64,CAPPED_NET_AMOUNT:float64,PO_QUAN:float64,CPD_UPDAT:float64", "tasks.max": "1", "topics": "DFP.HANA.SAPPRD.SAPABAP1.VBAP", "transforms": "cast", "DFP.HANA.SAPPRD.SAPABAP1.VBAP.incrementing.column.name": "KAFKA_ID", "mode": "incrementing", "connection.user": "BICONN", "DFP.HANA.SAPPRD.SAPABAP1.VBAP.table.name": "\"SAPABAP1\".\"VBAP_KAFKA\"", "name": "DFP.HANA.SAPPRD.SAPABAP1.VBAP", "connection.url": "jdbc:sap://10.115.49.33:30013?databaseName=HAP" }

the number in the table is 19980000 in the topic it is 1.998E7 How can I get the right value in the topic?Thanks so much.

elakito commented 2 years ago

@lawrenceee1 If the value is converted to float64 (that is double), its default print output representing 19980000 would be 1.998E7.

System.out.println("Output: " + (double)19980000);

Output: 1.998E7

If this column has no scale and you want its int64 value, you could use int64 as the target type.

Alternatively, you could wait for numeric.mapping support (https://github.com/SAP/kafka-connect-sap/issues/104).

elakito commented 2 years ago

Hi @lawrenceee1, As mentioned, if you want to have the float64 values, as configured in your Cast transformer's configuration, what you are getting in the records are correct. If you want to get the int64 values, either you need to configure your Cast transformer's configuration to use int64 or if your numerical types have all scale=0, you can use numeric.mapping=best_fit to map all those values into some intxx values that can be printed as integer.

I hope this answered your question and I am closing this ticket. If you have further questions related to this topic, please reopen it. regards, aki