Open wyz0130 opened 6 months ago
doris 2.0.9 2.1.2
使用 OUTFILE 导出到hdfs 上面,设置session变量set enable_parallel_outfile = true; 可以正常并发导出,设置success_file_name 后,hdfs的文件还没有导出完成,success_file就已经生成好了
如何导出完成后 在生成这个成功文件
1、set enable_parallel_outfile = true; 2、set parallel_fragment_exec_instance_num = 3; 3、创表 CREATE TABLE test1 ( id int(11) NULL, name varchar(500) NULL, age DECIMAL(10, 2) NULL, address boolean NULL, create_date date NULL, update_date datetime NULL ) ENGINE=OLAP DUPLICATE KEY(id) COMMENT 'OLAP' DISTRIBUTED BY HASH(id) BUCKETS 10 PROPERTIES ( "replication_allocation" = "tag.location.default: 1", "is_being_synced" = "false", "storage_format" = "V2", "light_schema_change" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false" );
test1
id
name
age
address
create_date
update_date
4、导出 SELECT * FROM test1 INTO OUTFILE "hdfs://192.168.31.129:9820/doris/" FORMAT AS CSV PROPERTIES ( "fs.defaultFS" = "hdfs://192.168.31.129:9820", "hadoop.username" = "wyz", "column_separator" = ",", "success_file_name" = "c1", "line_delimiter" = "\n" ); 5、结果
No response
测试的 2.1.3也有这个问题
Search before asking
Version
doris 2.0.9 2.1.2
What's Wrong?
使用 OUTFILE 导出到hdfs 上面,设置session变量set enable_parallel_outfile = true; 可以正常并发导出,设置success_file_name 后,hdfs的文件还没有导出完成,success_file就已经生成好了
What You Expected?
如何导出完成后 在生成这个成功文件
How to Reproduce?
1、set enable_parallel_outfile = true; 2、set parallel_fragment_exec_instance_num = 3; 3、创表 CREATE TABLE
test1
(id
int(11) NULL,name
varchar(500) NULL,age
DECIMAL(10, 2) NULL,address
boolean NULL,create_date
date NULL,update_date
datetime NULL ) ENGINE=OLAP DUPLICATE KEY(id
) COMMENT 'OLAP' DISTRIBUTED BY HASH(id
) BUCKETS 10 PROPERTIES ( "replication_allocation" = "tag.location.default: 1", "is_being_synced" = "false", "storage_format" = "V2", "light_schema_change" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false" );4、导出 SELECT * FROM test1 INTO OUTFILE "hdfs://192.168.31.129:9820/doris/" FORMAT AS CSV PROPERTIES ( "fs.defaultFS" = "hdfs://192.168.31.129:9820", "hadoop.username" = "wyz", "column_separator" = ",", "success_file_name" = "c1", "line_delimiter" = "\n" ); 5、结果
Anything Else?
No response
Are you willing to submit PR?
Code of Conduct