[Fix-15961] [UI] Rename pythonCommand to pythonLauncher in pytorch task.

cncws commented 1 month ago

Purpose of the pull request

This pull request fix #15961

Brief change log

Rename pythonCommand to pythonLauncher which is used in backend.

Verify this pull request

We can test on the running DS.

Case 1: set pythonCommand=/usr/bin/python3 in frontend, the value is set to pythonLauncher as following. Meanwhile, PYTHON_LAUNCHER is set in environment and point to a different python command. In the final shell, the pythonCommand is used as expected.

...
"taskParams" : "{\"localParams\":[],\"resourceList\":[],\"script\":\"main.py\",\"pythonPath\":\"/home/cws/hello-python\",\"pythonLauncher\":\"/usr/bin/python3\",\"pythonEnvTool\":\"conda\",\"requirements\":\"requirements.txt\",\"condaPythonVersion\":\"3.7\"}",
  "environmentConfig" : "export PYTHON_LAUNCHER=/opt/miniconda3/bin/python\nexport PATH=/opt/miniconda3/bin:$PATH",
...
[INFO] 2024-08-22 11:27:45.888 +0800 - Final Shell file is : 
#!/bin/bash
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
export PYTHON_LAUNCHER=/opt/miniconda3/bin/python
export PATH=/opt/miniconda3/bin:$PATH
export PYTHONPATH=/home/cws/hello-python
/usr/bin/python3 /home/cws/hello-python/main.py
[INFO] 2024-08-22 11:27:45.888 +0800 - Executing shell command : sudo -u cws -i /tmp/dolphinscheduler/exec/process/default/14297636570304/14297667467712_11/12/47/12_47.sh
[INFO] 2024-08-22 11:27:45.894 +0800 - process start, process id is: 6407
[INFO] 2024-08-22 11:27:46.897 +0800 -  -> 
    解释器 /usr/bin/python3
    版本 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]

Case 2: pythonCommand is set, $PYTHON_LAUNCHER not exists.

...
"taskParams" : "{\"localParams\":[],\"resourceList\":[],\"script\":\"main.py\",\"pythonPath\":\"/home/cws/hello-python\",\"pythonLauncher\":\"/usr/bin/python3\",\"pythonEnvTool\":\"conda\",\"requirements\":\"requirements.txt\",\"condaPythonVersion\":\"3.7\"}",
...
[INFO] 2024-08-22 11:27:45.894 +0800 - Final Shell file is : 
#!/bin/bash
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
export PYTHONPATH=/home/cws/hello-python
/usr/bin/python3 /home/cws/hello-python/main.py
[INFO] 2024-08-22 11:27:45.894 +0800 - Executing shell command : sudo -u cws -i /tmp/dolphinscheduler/exec/process/default/14297636570304/14297667467712_11/12/50/12_50.sh
[INFO] 2024-08-22 11:27:45.899 +0800 - process start, process id is: 6413
[INFO] 2024-08-22 11:27:46.902 +0800 -  -> 
    解释器 /usr/bin/python3
    版本 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]

Case 3: pythonCommand is not set, use default $PYTHON_LAUNCHER.

"taskParams" : "{\"localParams\":[],\"resourceList\":[],\"script\":\"main.py\",\"pythonPath\":\"/home/cws/hello-python\",\"pythonLauncher\":\"\",\"pythonEnvTool\":\"conda\",\"requirements\":\"requirements.txt\",\"condaPythonVersion\":\"3.7\"}",
  "environmentConfig" : "export PYTHON_LAUNCHER=/opt/miniconda3/bin/python\nexport PATH=/opt/miniconda3/bin:$PATH",
...
[INFO] 2024-08-22 11:27:45.886 +0800 - Final Shell file is : 
#!/bin/bash
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
export PYTHON_LAUNCHER=/opt/miniconda3/bin/python
export PATH=/opt/miniconda3/bin:$PATH
export PYTHONPATH=/home/cws/hello-python
${PYTHON_LAUNCHER} /home/cws/hello-python/main.py
[INFO] 2024-08-22 11:27:45.888 +0800 - Executing shell command : sudo -u cws -i /tmp/dolphinscheduler/exec/process/default/14297636570304/14297667467712_11/12/49/12_49.sh
[INFO] 2024-08-22 11:27:45.897 +0800 - process start, process id is: 6406
[INFO] 2024-08-22 11:27:46.901 +0800 -  -> 
    解释器 /opt/miniconda3/bin/python
    版本 3.11.5 (main, Sep 11 2023, 13:14:08) [GCC 11.2.0]