[Bug] StartupParams has lost in CodeLogicalUnitExecTask.toRequestTask method

[fjibj]

Search before asking

• [X] I searched the issues and found no similar issues.

Linkis Component

linkis-engineconnn-plugin, linkis-orchestrator

Steps to reproduce

Step 1: Add some attributes such as "jdbc.url" for workflow JDBC node, just insert some key in dss_workflow_node_ui table. You can see a new input appear in Attribute information of jdbc. So you can get the request json include "jdbc.url" in startup params.

params: {variable: {}, configuration: {special: {}, runtime: {,…}, startup: {ReuseEngine: "true",…}}} configuration: {special: {}, runtime: {,…}, startup: {ReuseEngine: "true",…}} runtime: {,…} contextID: "{"type":"HAWorkFlowContextID","value":"{\"instance\":null,\"backupInstance\":null,\"user\":\"root\",\"workspace\":\"bdapWorkspace\",\"project\":\"test22\",\"flow\":\"t22flow1\",\"contextId\":\"8-8--cs_1_devcs_1_dev20\",\"version\":\"v000001\",\"env\":\"BDAP_DEV\"}"}" nodeName: "jdbc_3011" special: {} startup: {ReuseEngine: "true",…} ReuseEngine: "true" jdbc.url: "jdbc:mysql://xx.xx.xx.xx:yyyy/mysql" variable: {}

Step 2: But you can't get the value of "jdbc.url" params from the properties of engineExecutorContext in EngineExecutor classes , such as JDBCEngineConnExecutor.scala, So, it must lost during the transmission process.

Expected behavior

I expect to get the value of startup params in the properties of engineExecutorContext in any EngineExecutor classes, or give me some other method to get them,maybe from JobHistory.

But in my opinion, it is a public problem that needs to be fixed. I checked the code and found it lost in CodeLogicalUnitExecTask.toRequestTask method that passed a lot of parames except startup.

I added some code in CodeLogicalUnitExecTask.toRequestTask method as an interim proposal // add by fangjin 20230210 if (null != getParams.getStartupParams) { requestTask.getProperties.putAll(getParams.getStartupParams.getConfigurationMap) } //

Your environment

• Linkis version used: 1.3.1
• Environment name and version:
  • hadoop-3.2.1
  • hive-3.1.2
  • spark-2.4.6
  • scala-2.11.12
  • jdk 1.8.0_202
  • ....

Anything else

No response

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR!

[github-actions[bot]]

:blush: Welcome to the Apache Linkis community!!

We are glad that you are contributing by opening this issue.

Please make sure to include all the relevant context. We will be here shortly.

If you are interested in contributing to our website project, please let us know! You can check out our contributing guide on :point_right: How to Participate in Project Contribution.

Community

WeChat Assistant	WeChat Public Account

Mailing Lists

Name	Description	Subscribe	Unsubscribe	Archive
dev@linkis.apache.org	community activity information	subscribe	unsubscribe	archive

[peacewong]

Thank you for your explanation, this should be a DSS bug. Because StartUp should be the parameter used when starting Ec. You can refer to: org.apache.linkis.orchestrator.computation.execute.DefaultCodeExecTaskExecutorManager#createMarkReq The URL of JDBC should be put into Runtime parmas. But I think it can be compatible here, but in order to prevent parameter duplication, it is recommended to put startUP first and then put runtime

[casionone]

I will close this issue. You can reopen if necessary