search

practo / tipoca-stream

Near real time cloud native data pipeline in AWS (CDC+Sink). Hosts code for RedshiftSink. RDS to RedshiftSink Pipeline with masking and reloading support.
https://towardsdatascience.com/open-sourcing-tipoca-stream-f261cdcc3a13
Apache License 2.0
47 stars 5 forks source link

Handle Float Unsigned Source Type #267

Closed mayankdharwa closed 2 years ago

mayankdharwa commented 2 years ago

Fixes: https://practo.pagerduty.com/incidents/Q3HME1DNC7FE8I

Evidence [Pod Logs]:

E0803 11:20:38.354942       1 batch_processor.go:609] ts.reach_new.unit_price_mapping, error(s) occured: 3 errors occurred:
    * transforming schema:1625 => inputTable failed: DebeziumType: double, SourceType: float unsigned, not handled

    * transforming schema:1625 => inputTable failed: DebeziumType: double, SourceType: float unsigned, not handled

    * transforming schema:1625 => inputTable failed: DebeziumType: double, SourceType: float unsigned, not handled

, processor shutdown.
E0803 11:20:38.355063       1 batcher_handler.go:272] consumeClaim returning, ts.reach_new.unit_price_mapping: error occured in processing, err: 3 errors occurred:
    * transforming schema:1625 => inputTable failed: DebeziumType: double, SourceType: float unsigned, not handled

    * transforming schema:1625 => inputTable failed: DebeziumType: double, SourceType: float unsigned, not handled

    * transforming schema:1625 => inputTable failed: DebeziumType: double, SourceType: float unsigned, not handled

, triggered shutdown
[sarama] 2022/08/03 11:20:38 loop check partition number coroutine will exit, topics [ts.reach_new.unit_price_mapping]