Numeric column as Protobuf uint64/uint32

[silentsokolov]

Describe the bug

Cannot create a sink form a source with identical protobuf shema

Error message/log

[2024-11-21 12:26:42] [XX000] ERROR: Failed to execute the statement
[2024-11-21 12:26:42] Caused by these errors (recent errors listed first):
[2024-11-21 12:26:42] 1: gRPC request to meta service failed: Internal error
[2024-11-21 12:26:42] 2: failed to validate sink
[2024-11-21 12:26:42] 3: Encode error: encode 'uid' error: cannot encode numeric column as uint64 field

[2024-11-21 12:52:08] Caused by these errors (recent errors listed first):
[2024-11-21 12:52:08] 1: gRPC request to meta service failed: Internal error
[2024-11-21 12:52:08] 2: failed to validate sink
[2024-11-21 12:52:08] 3: Encode error: encode 'uid' error: cannot encode bigint column as uint32 field

To Reproduce

message Fact {
  string id = 1;
  uint64 uid = 2; // or uint32
}

protoc --include_imports --descriptor_set_out=schema.pb schema.proto 

CREATE SOURCE source_test
WITH (
   connector='kafka'
) FORMAT PLAIN ENCODE PROTOBUF (
   message='Fact',
   schema.location='file:/// ..schema.pb'
);

CREATE MATERIALIZED VIEW mv_test AS
SELECT
    *
FROM
    source_test;

CREATE SINK sink_test
FROM mv_test
WITH (
   connector='kafka'
) FORMAT PLAIN ENCODE PROTOBUF (
   message='Fact',
   schema.location='file:/// ..schema.pb'
);

Expected behavior

I expected that the sink created

How did you deploy RisingWave?

Homebrew

The version of RisingWave

PostgreSQL 13.14.0-RisingWave-2.0.2 (Homebrew)

Additional context

No response

[xiangjinwu]

This is because there is no unsigned data types in RisingWave, so we have been conservative on implicit conversion:

• In source, uint32 is promoted to bigint and uint64 is promoted to numeric (aka decimal).
• In sink, bigint may not fit back into a uint32, similarly numeric into uint64.

To allow such round trips, we may require an explicit acknowledgement such as allow_lossy_cast = true (name TBD). Better design alternatives are welcomed.