search

rapidsai / distributed-join

Other
19 stars 12 forks source link

Initial CI for project #47

Closed esoha-nvidia closed 3 years ago

esoha-nvidia commented 3 years ago

This script expects that WEBHOOK_URL, CONDA_PREFIX, and CI_DIR are defined in the environment.

WEBHOOK_URL should be of the form: WEBHOOK_URL=https://hooks.slack.com/services/TXXXXBXXXX/XXXXX

CONDA_PREFIX should point to the directory where conda was installed.

CI_DIR should point to a path that already exists and has space enough to install the code. It will be cleaned after each run. If not specified, the current directory is used.

Concurrent runs of the script are possible.

Example usage in crontab:

26 bash -c 'source /etc/profile; git -C /home/$USER/distributed-join-ci pull && env WEBHOOK_URL=WEBHOOK_URL=https://hooks.slack.com/services/TXXXXBXXXX/XXXXX CONDA_PREFIX=/home/$USER/miniconda2 CI_DIR=/home/$USER/ci sbatch --time=30 --nodes=4 --ntasks-per-node=8 -W /home/$USER/distributed-join-ci/ci/ci.sh'

gaohao95 commented 3 years ago

Overall looks good to me. Thanks for working on this. An additional comment:

26 bash -c 'source /etc/profile; git -C /home/$USER/distributed-join-ci pull && env WEBHOOK_URL=WEBHOOK_URL=https://hooks.slack.com/services/TXXXXBXXXX/XXXXX CONDA_PREFIX=/home/$USER/miniconda2 CI_DIR=/home/$USER/ci sbatch --time=30 --nodes=4 --ntasks-per-node=8 -W /home/$USER/distributed-join-ci/ci/ci.sh'

Just FYI, this script will only work on a single node, so we should use 1 for --nodes.