Bartender is a middleware web service to schedule jobs on various infrastructures.
It can run command line applications for visitors. The application input should be a configuration file with links to data files in the same directory. After acquiring a JWT token, you can upload your configuration file and data files as an archive to the web service for submission. Once the job has been executed the output files can be browsed with a web browser.
Bartender can be configured to run applications on a Slurm batch scheduler, pilot job framework, the grid or in the cloud. Bartender will take care of moving the input and output files to the right place. To pick where an application should be run you can choose from a list of existing Python functions or supply your own.
Bartender can run quick interactive applications on completed jobs. This is handy if you want to run a quick analysis on the output of a job.
Bartender can be used as the computational backend for a web application, the web application should guide visitors into the submission and show the results. See https://github.com/i-VRESSE/haddock3-webapp for an example.
Documentation for users and developers is available at https://i-vresse-bartender.readthedocs.io .
Install bartender via GitHub:
# Inside an empty directory
git clone https://github.com/i-VRESSE/bartender.git .
pip install .
Obtain a copy of the example configuration file
cp config-example.yaml config.yaml
In another terminal, start up a database for storing jobs.
docker run \
-p "5432:5432" \
-e "POSTGRES_PASSWORD=bartender" \
-e "POSTGRES_USER=bartender" \
-e "POSTGRES_DB=bartender" \
--mount type=volume,source=bartender-db,target=/var/lib/postgresql/data \
postgres:15.2-bullseye
(Use docker volume rm bartender-db
to clear the database storage`)
Create tables in the database
alembic upgrade "head"
Generate token to authenticate yourself
# Generate a rsa key pair
openssl genpkey -algorithm RSA -out private_key.pem \
-pkeyopt rsa_keygen_bits:2048
openssl rsa -pubout -in private_key.pem -out public_key.pem
bartender generate-token --username myname
Run the application
bartender serve
Go to the interactive API documentation generated by FastAPI
The interactive API documentation generated by FastAPI is at http://localhost:8000/api/docs
To consume the bartender web service you need to authenticate yourself with a JWT token in the
Authorization: Bearer <token>
or?token=<token>
orbartenderToken=<token>
of the HTTP request.For more info see Configuration docs
Bartender is by default configured with a word count applicaton. Use the following steps to run a job:
README.md
. A zip file could be created in a clone of this repo with
zip README.zip README.md
.bartender generate-token
and copy output to clipboard.
(Make sure to not to have any newlines in the token)http://127.0.0.1:8000/api/docs
andGET /api/whoami
route.
It should return JSON document with someone
as username.PUT /api/application/wc
route.README.zip
as request body.id
property) that can be used
to fetch the job state.GET /api/job/{jobid}
jobid
parameter
value.
ok
the job was completed succesfully.wc
) outputs to the stdout.
GET /api/job/{jobid}/stdout
jobid
parameter
value.433 1793 14560 README.md
. Where numbers are
counts for newlines, words, bytes.