[Bug] Can not insert data after enabling the partial column updates

[bingwill]

Search before asking

• [X] I had searched in the issues and found no similar issues.

Version

both 2.0.2 and 2.0.3

What's Wrong?

It is deadly wierd that I can not insert data after Setting up partial column updates both on version 2.0.2 and 2.0.3,.And of cause I get the error message :Insert has filtered data in strict mode, tracking_url=null when inserting data whch was not existed in table.If insert an existing data it would be very smoothly.And when setting up the session variables enable_unique_key_partial_update to false it can be proceeded without any error.

table stucture: CREATE TABLE test_table ( sdv_id int(11) NOT NULL, im_id varchar(36) NULL, series_type varchar(50) NULL, thing_category varchar(50) NULL, create_time datetime NULL ) ENGINE=OLAP UNIQUE KEY(sdv_id) COMMENT 'OLAP' DISTRIBUTED BY HASH(sdv_id) BUCKETS 2 PROPERTIES ( "replication_allocation" = "tag.location.default: 1", "is_being_synced" = "false", "storage_format" = "V2", "enable_unique_key_merge_on_write" = "true", "light_schema_change" = "true", "store_row_column" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false" );

sql: set enable_unique_key_partial_update=true; insert into iotdata.test_true (sdv_id,im_id,series_type,thing_category,create_time) values (6,'im001','60','AGC','2023-11-22 00:00:00'), (7,'im001','60','AGC','2023-11-22');

I can not find any useful info both in fe's and be's log but:Insert has filtered data in strict mode, tracking_url=null. The columns and values arre match i suppose.I ran into the same error before,but they were all stirred up by type,length of colum or sth like this,but this time war different. Is thre any thing differently? If any one can help me ,it would be very considerate. thanks a lot.

What You Expected?

1.insert data without any error. 2.If there were errors could it would be very kindly to offer more information but not only the Insert has filtered data in strict mode, tracking_url=null witch linked to a null url that was not helpful sometines.☺ Thank.

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[liugddx]

Please refer to https://doris.apache.org/zh-CN/docs/dev/data-operate/update-delete/partial-update?_highlight=enable_unique_key_partial_update#%E9%80%82%E7%94%A8%E5%9C%BA%E6%99%AF

[liugddx]

Please refer to https://doris.apache.org/zh-CN/docs/dev/data-operate/update-delete/partial-update?_highlight=enable_unique_key_partial_update#%E9%80%82%E7%94%A8%E5%9C%BA%E6%99%AF

set enable_insert_strict = false;

[bingwill]

Please refer to https://doris.apache.org/zh-CN/docs/dev/data-operate/update-delete/partial-update?_highlight=enable_unique_key_partial_update#%E9%80%82%E7%94%A8%E5%9C%BA%E6%99%AF

set enable_insert_strict = false;

Thank you. Yes, I've noticed this enable_insert_strict too,but it seems to be focused on those column keys that do not exist int table only.In this case,all my key match.So it might be a diffrent situation.And most import is if i set enable_insert_strict to false ,doris may return succese when one entry imported correctly according to the docs. . What i want is either all data import correctly or no one import correctly.

[liugddx]

Because you have some data that does not exist in the table, in unique-key table you need set enable_insert_strict = false. I tested using your table and everything is fine

[bobhan1]

It seems that columns in the insert statement covers all the columns in the table, so you don't need to set enable_unique_key_partial_update to true. The upsert semantic of MOW unique table can satisfy your requirements.

[bingwill]

It seems that columns in the insert statement covers all the columns in the table, so you don't need to set enable_unique_key_partial_update to true. The upsert semantic of MOW unique table can satisfy your requirements.

Do not cover all the columns this time,and error occurs still.

[bobhan1]

Because there is no row with sdv_id=666 in the original table, in partial updates, this newly inserted row will be counted in the filtered rows. Then, due to enable_insert_strict=true, as long as there are filtered rows, the entire load will fail.

[bingwill]

Thank you for your kindly explanation,and now i know what it really is.But i still think it can not be called the upsert.It might be a little bit arbitraryto count this newly inserted row int the filtered rows as no unqualified data quality was found there.

[bingwill]

Because you have some data that does not exist in the table, in unique-key table you need set enable_insert_strict = false. I tested using your table and everything is fine

tks