install problem

[summerghw]

Traceback (most recent call last):
  File "./cwl-airflow", line 43, in <module>
    sys.exit(main(sys.argv[1:]))
  File "./cwl-airflow", line 39, in main
    args.func(args)
  File "./cwl-airflow", line 12, in run_init
    launcher.init()
  File "/home/ds/miniconda3/lib/python3.7/site-packages/cwl_airflow/app/launch.py", line 53, in init
    self.init_airflow_db()
  File "/home/ds/miniconda3/lib/python3.7/site-packages/cwl_airflow/app/launch.py", line 156, in init_airflow_db
    subprocess.run(["airflow", "initdb"], env=env)
  File "/home/ds/miniconda3/lib/python3.7/subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/home/ds/miniconda3/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/home/ds/miniconda3/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'

hello, when i try to install cwl-airflow, this problem occur. i pre install airflow independently, does this the reason?

[michael-kotliar]

Hello,

Thank you for using CWL-Airflow. Could you please provide some additional information such as your OS, CWL-Airflow version, or GitHub commit you have installed our program from. Also, from the attached log I can see that you were trying to run everything in miniconda, which might have changed some of your environment variables. Would you mind running the export command to see what environment variables had been changed? Additionally, you mentioned that you have already pre-installed Airflow. Were you able to successfully run airflow --help, airflow initdb, airflow scheduler or airflow webserver?

Thanks, Michael

[summerghw]

@michael-kotliar thank you for reply, my OS is Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-55-generic x86_64) CWL-Airflow was latest , airflow version was 1.10.10 environment variables was

`declare -x CONDA_DEFAULT_ENV="base"
declare -x CONDA_EXE="/home/ds/miniconda3/bin/conda"
declare -x CONDA_PREFIX="/home/ds/miniconda3"
declare -x CONDA_PROMPT_MODIFIER="(base) "
declare -x CONDA_PYTHON_EXE="/home/ds/miniconda3/bin/python"
declare -x CONDA_SHLVL="1"
declare -x DISPLAY="localhost:10.0"
declare -x HOME="/home/ds"
declare -x JAVA_HOME="/home/ds/miniconda3"
declare -x JAVA_HOME_CONDA_BACKUP=""
declare -x JAVA_LD_LIBRARY_PATH="/home/ds/miniconda3/jre/lib/amd64/server"
declare -x JAVA_LD_LIBRARY_PATH_BACKUP=""
declare -x LANG="en_US.UTF-8"
declare -x LESSCLOSE="/usr/bin/lesspipe %s %s"
declare -x LESSOPEN="| /usr/bin/lesspipe %s"
declare -x LOGNAME="ds"
declare -x LS_COLORS="rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:"
declare -x MAIL="/var/mail/ds"
declare -x OLDPWD
declare -x PATH="/home/ds/.local/bin:/home/ds/miniconda3/bin:/home/ds/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/ds/tools/sratoolkit.2.8.2-centos_linux64/bin"
declare -x PWD="/home/ds"
declare -x SHELL="/bin/bash"
declare -x SHLVL="1"
declare -x SSH_CLIENT="10.1.23.64 57341 22"
declare -x SSH_CONNECTION="10.1.23.64 57341 192.168.99.102 22"
declare -x SSH_TTY="/dev/pts/1"
declare -x S_COLORS="auto"
declare -x TERM="xterm"
declare -x TZ="Asia/Shanghai"
declare -x USER="ds"
declare -x XDG_DATA_DIRS="/usr/local/share:/usr/share:/var/lib/snapd/desktop"
declare -x XDG_RUNTIME_DIR="/run/user/1000"
declare -x XDG_SESSION_ID="13499"
declare -x _CE_CONDA=""
declare -x _CE_M=""

I can run airflow successfully. i tried to change AIRFLOW_HOM from ~/airflow to conda/bin , it does not work.

[michael-kotliar]

Ah, ok, I see. I think in your case you don't even need to run cwl-airflow init. If you have your airflow already installed and configured, you just need to put a new dag into your dag folder and a run airflow scheduler and webserver.

Example of dag file. Update the path to workflow and dag_id

#!/usr/bin/env python3
from cwl_airflow import CWLDAG, CWLJobDispatcher, CWLJobGatherer
dag = CWLDAG(cwl_workflow="/absolute/path/to/your/packed/workflow.cwl", dag_id="any unique id")
dag.create()
dag.add(CWLJobDispatcher(dag=dag), to='top')
dag.add(CWLJobGatherer(dag=dag), to='bottom')

[summerghw]

@michael-kotliar thank you a lot.