apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.26k stars 3.2k forks source link

[Bug] OUTFILE 的success_file_name 导出未完成就生成了success_file文件 #34567

Open wyz0130 opened 3 months ago

wyz0130 commented 3 months ago

Search before asking

Version

doris 2.0.9 2.1.2

What's Wrong?

使用 OUTFILE 导出到hdfs 上面,设置session变量set enable_parallel_outfile = true; 可以正常并发导出,设置success_file_name 后,hdfs的文件还没有导出完成,success_file就已经生成好了

What You Expected?

如何导出完成后 在生成这个成功文件

How to Reproduce?

1、set enable_parallel_outfile = true; 2、set parallel_fragment_exec_instance_num = 3; 3、创表 CREATE TABLE test1 ( id int(11) NULL, name varchar(500) NULL, age DECIMAL(10, 2) NULL, address boolean NULL, create_date date NULL, update_date datetime NULL ) ENGINE=OLAP DUPLICATE KEY(id) COMMENT 'OLAP' DISTRIBUTED BY HASH(id) BUCKETS 10 PROPERTIES ( "replication_allocation" = "tag.location.default: 1", "is_being_synced" = "false", "storage_format" = "V2", "light_schema_change" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false" );

4、导出 SELECT * FROM test1 INTO OUTFILE "hdfs://192.168.31.129:9820/doris/" FORMAT AS CSV PROPERTIES ( "fs.defaultFS" = "hdfs://192.168.31.129:9820", "hadoop.username" = "wyz", "column_separator" = ",", "success_file_name" = "c1", "line_delimiter" = "\n" ); 5、结果 7df9c52831a1e1109f60786c24a2506

Anything Else?

No response

Are you willing to submit PR?

Code of Conduct

wyz0130 commented 3 months ago

测试的 2.1.3也有这个问题