search

risingwavelabs / risingwave

SQL stream processing, analytics, and management. We decouple storage and compute to offer instant failover, dynamic scaling, speedy bootstrapping, and efficient joins.
https://www.risingwave.com/slack
Apache License 2.0
6.59k stars 538 forks source link

bug(cdc): wrong output when mapping pg-numeric to varchar in postgres-cdc #16395

Open KeXiangWang opened 2 months ago

KeXiangWang commented 2 months ago

Describe the bug

In https://github.com/risingwavelabs/risingwave/pull/16346, we allow mapping numeric to varchar in postgres-cdc to avoid the precision loss of pg-numeric to rw-numeric in some scenarios. However the spacial value are converted to wrong string. To be specific:

value current output string expected output string
nan NAN NaN
inf POSITIVE_INFINITIY Infinity
-inf NEGATIVE_INFINITIY -Infinity

See more discussion here.

Error message/log

No response

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

No response

xiangjinwu commented 2 months ago

Caused by Debezium's PostgreSQL connector: https://gist.github.com/xiangjinwu/e879234981b65b08a074f27f6bab54ff?permalink_comment_id=5033378#gistcomment-5033378

StrikeW commented 2 months ago

The default value of decimal.handling.mode is string in our config. https://github.com/risingwavelabs/risingwave/blob/201f90301041657d47a88ff3560febde16d4f123/java/connector-node/risingwave-connector-service/src/main/resources/debezium.properties#L10

Not sure whether our DebeziumJsonParser can handle the precise mode

precise represents values by using java.math.BigDecimal to represent values in binary form in change events.