StarRocks discards the NOT NULL constraint of the external table when constructing the full schema, leading to the failure of view creation

[elon-X]

When I try to build an asynchronous partitioned view based on an external table (paimon table / hive table) with a partition field (string type), I encounter an error: ERROR 1064 (HY000): List partition columns must not be nullable in Materialized view for now, which causes the asynchronous view construction to fail.

Root Cause:

1.StarRocks checks whether partition fields have the NOT NULL constraint when building asynchronous views.

2.Upon checking the schema of the paimon table, the partition field does have the NOT NULL constraint.

3.When StarRocks constructs the full schema, it forces all fields to be nullable. This results in an exception being thrown when constructing the view

Additionally, this issue can be observed by using DESC tableA; or SHOW CREATE TABLE tableA;

Steps to reproduce the behavior (Required)

1. desc tableA; // show create table tableA;

Expected behavior (Required)

Real behavior (Required)

StarRocks version (Required)

StarRocks-3.3.0

[LiShuMing]

Materialized view support list partition with nullable column will be supported in 3.3.1/3.3.2.

[elon-X]

@LiShuMing I checked the code in the main branch, and when constructing the fullSchema, all fields are still set to isAllowNull = true. The isNullable field in DataType can obtain the non-null constraints from the original schema, so we can set this value instead of hardcoding it to true. This will cause the table schema information displayed by different clients to be inconsistent, which will confuse users. WDYT?

Column column = new Column(fieldName, fieldType, type.isNullable(), field.description());

[LiShuMing]

@elon-X Yep, you are right. But for external tables, not null constraint maybe exactly right since its strong schema evolution.

You can try StarRocks v3.3.2 again, this issue should be fixed already.