search

apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
https://dolphinscheduler.apache.org/
Apache License 2.0
12.8k stars 4.6k forks source link

[Question] sqoop脚本运行成功，状态显示失败 #3851

Closed 1015zhc closed 3 years ago

1015zhc commented 4 years ago

For better global communication, please give priority to using English description, thx!

Please review https://dolphinscheduler.apache.org/en-us/docs/development/issue.html when describe an issue.

Describe the question A clear and concise description of what the question is. 当执行sqoop节点脚本时，查看日志执行成功，状态显示失败。 查看日志，并无报错。 image

Which version of DolphinScheduler: -[1.3.2]

Additional context Add any other context about the problem here.

Requirement or improvement

1015zhc commented 4 years ago

I ask others ,they say that maybe 'the numeric of app_link is too short in the table of t_ds_task_instance'; but the type is changed text in 1.3.2;

Eights-Li commented 4 years ago

please check the task log in worker-server, this maybe yarn_ips conf wrong. can you show the sqoop task log in ds?

1015zhc commented 4 years ago

please check the task log in worker-server, this maybe yarn_ips conf wrong. can you show the sqoop task log in ds?

Is yarn_ips conf in conf/config/install_config.conf? I have not conf it. OK ,I have a try. image

geosmart commented 4 years ago

after the sqoop yarn task finish, ds will check the yarn job info by rest api.

if your yarn config in ds is not correct, the task res will not success

# if resourcemanager HA enable, please type the HA ips ; if resourcemanager is single, make this value empty
yarn.resourcemanager.ha.rm.ids=ip1,ip2
resource.manager.httpaddress.port=8088
# if resourcemanager HA enable or not use resourcemanager, please keep the default value; If resourcemanager is single, you only need to replace ds1 to actual resourcemanager hostname.
yarn.application.status.address=http://xxx:8088/ws/v1/cluster/apps/%s

xxx in yarn.application.status.address will replaced by active yarn resource manager in yarn.resourcemanager.ha.rm.ids

when master stop the sqoop process ,the kill command send to the worker server, so the worker will kill the yarn job by rest api

xingchun-chen commented 4 years ago

please update the title in English

xingchun-chen commented 3 years ago

The title has not been updated for a long time in English , it will be closed. If you have any questions, please update the title in English and reopen it.

qulibibg commented 3 years ago

我也出现这个问题，ResourceManager是单节点，最近考hadoop的配置文件的yarn的地址，多了端口号，singleYarnIp不应该出现端口号

1015zhc commented 3 years ago

是的，我配置后正常了 image