apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
https://dolphinscheduler.apache.org/
Apache License 2.0
12.72k stars 4.58k forks source link

The parameter data format cannot be changed when assembling SQL query parameters. When parameter variables are added to the create table name and used, the " signs" on both sides of the variables cannot be removed. #16344

Open kieryum opened 2 months ago

kieryum commented 2 months ago

Search before asking

What happened

错误: [INFO] 2024-07-18 03:25:56.962 -0400 - prepare statement replace sql : create table xxxx? PRIMARY KEY keys as select ?,? as keys, sql parameters : {1=Property{prop='table_name', direct=IN, type=VARCHAR, value='123'}, 2=Property{prop='var_1', direct=IN, type=VARCHAR, value='23'}, 3=Property{prop='var_2', direct=IN, type=VARCHAR, value='50'}} [ERROR] 2024-07-18 03:25:56.987 -0400 - execute sql error: Code: 62. DB::Exception: Syntax error: failed at position 20 (''123'') (line 1, col 20): '123' PRIMARY KEY keys as select '23','50' as keys. Expected one of: token, Dot, UUID, ON, OpeningRoundBracket, storage definition, ENGINE, PARTITION BY, PRIMARY KEY, ORDER BY, SAMPLE BY, TTL, EMPTY AS, AS, COMMENT, INTO OUTFILE, FORMAT, SETTINGS, end of query. (SYNTAX_ERROR) (version 24.5.3.5 (official build)) , server ClickHouseNode [uri=http://10.10.60.26:8123/default]@-366638818 [ERROR] 2024-07-18 03:25:57.432 -0400 - sql task error java.sql.BatchUpdateException: Code: 62. DB::Exception: Syntax error: failed at position 20 (''123'') (line 1, col 20): '123' PRIMARY KEY keys as select '23','50' as keys. Expected one of: token, Dot, UUID, ON, OpeningRoundBracket, storage definition, ENGINE, PARTITION BY, PRIMARY KEY, ORDER BY, SAMPLE BY, TTL, EMPTY AS, AS, COMMENT, INTO OUTFILE, FORMAT, SETTINGS, end of query. (SYNTAX_ERROR) (version 24.5.3.5 (official build))

我的目标: 在上游节点生成参数,在下游节点上通过上游的传参动态生成数据表。发现上游数据类型无论怎么变化,下游拿到的依然时字符串带双引号。 例如:我想生成脚本:create table xxxx_${table_name} as select xxxx ; 动态参数 ${table_name} 为 1001 或者 xx01 实际生成了create table xxxx'1001' as select xxxx ; 或者 create table xxxx_'xx_01' as select xxxx ; 无论上游的out 出的数据是 long 还是 interger 都无法改变现状

What you expected to happen

官方文档告诉这里是可用的。 当接收到上游的数据${table_name}=xx001 值后 , 下游在使用时可以 create table xxx${table_name} 可以生成 create table xx_001 ... 现实是生成了 create table 'xx_001' .

How to reproduce

目前只在sql 上出现,shell 中没有出现

Anything else

No response

Version

3.2.x

Are you willing to submit PR?

Code of Conduct

github-actions[bot] commented 2 months ago

Search before asking

What happened

错误: [INFO] 2024-07-18 03:25:56.962 -0400 - prepare statement replace sql : create table xxxx? PRIMARY KEY keys as select ?,? as keys, sql parameters : {1=Property{prop='table_name', direct=IN, type=VARCHAR, value='123'}, 2=Property{prop='var_1', direct=IN, type=VARCHAR, value='23'}, 3=Property{prop='var_2', direct=IN, type=VARCHAR, value='50'}} [ERROR] 2024-07-18 03:25:56.987 -0400 - execute sql error: Code: 62. DB::Exception: Syntax error: failed at position 20 (''123'') (line 1, col 20): '123' PRIMARY KEY keys as select '23','50' as keys. Expected one of: token, Dot, UUID, ON, OpeningRoundBracket, storage definition, ENGINE, PARTITION BY, PRIMARY KEY, ORDER BY, SAMPLE BY, TTL, EMPTY AS, AS, COMMENT, INTO OUTFILE, FORMAT, SETTINGS, end of query. (SYNTAX_ERROR) (version 24.5.3.5 (official build)) , server ClickHouseNode [uri=http://10.10.60.26:8123/default]@-366638818 [ERROR] 2024-07-18 03:25:57.432 -0400 - sql task error java.sql.BatchUpdateException: Code: 62. DB::Exception: Syntax error: failed at position 20 (''123'') (line 1, col 20): '123' PRIMARY KEY keys as select '23','50' as keys. Expected one of: token, Dot, UUID, ON, OpeningRoundBracket, storage definition, ENGINE, PARTITION BY, PRIMARY KEY, ORDER BY, SAMPLE BY, TTL, EMPTY AS, AS, COMMENT, INTO OUTFILE, FORMAT, SETTINGS, end of query. (SYNTAX_ERROR) (version 24.5.3.5 (official build))

我的目标: 在上游节点生成参数,在下游节点上通过上游的传参动态生成数据表。发现上游数据类型无论怎么变化,下游拿到的依然时字符串带双引号。 例如:我想生成脚本:create table xxxx_${table_name} as select xxxx ; 动态参数 ${table_name} 为 1001 或者 xx01 实际生成了create table xxxx'1001' as select xxxx ; 或者 create table xxxx_'xx_01' as select xxxx ; 无论上游的out 出的数据是 long 还是 interger 都无法改变现状

What you expected to happen

官方文档告诉这里是可用的。 当接收到上游的数据${table_name}=xx001 值后 , 下游在使用时可以 create table xxx${table_name} 可以生成 create table xx_001 ... 现实是生成了 create table 'xx_001' .

How to reproduce

目前只在sql 上出现,shell 中没有出现

Anything else

No response

Version

3.2.x

Are you willing to submit PR?

Code of Conduct

kieryum commented 2 months ago

测试了 3.2.1 3.2.0 都不行。直到3.1.9 是可用的