search

facebookarchive / bistro

Bistro is a flexible distributed scheduler, a high-performance framework supporting multiple paradigms while retaining ease of configuration, management, and monitoring.
https://bistro.io
MIT License
1.03k stars 158 forks source link

How send job to bistro_worker ? #41

Closed linuxpham closed 4 years ago

linuxpham commented 4 years ago

Hi admin,

I started : bistro_scheduler --server_port=6789 --http_server_port=6790 --config_file=${ROOT_PATH}/configs/bistro/scheduler.json --data_dir=/mnt/data/bistro/scheduler --clean_statuses --CAUTION_startup_wait_for_workers=1 --instance_node_name=scheduler

bistro_worker --server_port=27182 --scheduler_host=:: --scheduler_port=6789 --worker_command="${ROOT_PATH}/configs/bistro/demo_bistro_task.sh" --data_dir=/mnt/data/bistro/worker

At the moment, scheduler + worker servers are running in two ports: 6789 (scheduler), 27182 (worker). How do I post new job (with parameter) for bistro worker ?

Is bistro architecture the same Gearmand? (http://gearman.org/)

Thank you!

snarkmaster commented 4 years ago

Your --config_file is supposed to list all jobs that you want to run. This file gets re-read periodically.

Take a look at the job definition in scripts/test_configs/simple mentioned here: https://facebook.github.io/bistro/docs/getting-started/

Production use-cases typically read from some kind of config storage service instead of a JSON file (unfortunately, the FB production services are not open-source, so those plugins are not available. Writing a plugin for another config source is pretty easy, just follow FileConfigLoader.{h,cpp}.

Bistro's architecture / use-case is fairly different from that of Gearman. I'd suggest reviewing the USENIX paper linked from https://bistro.io/ and the docs to get a better idea.

snarkmaster commented 4 years ago

(please re-open if needed)