saga-project / BigJob

SAGA-based Pilot-Job Implementation for Compute and Data
http://saga-project.github.com/BigJob/
Other
19 stars 8 forks source link

Allow for default coordination mechanism #6

Closed oweidner closed 10 years ago

oweidner commented 12 years ago

Especially with BigJob deployment on XSEDE and the fact that we decided to move to Redis, it doesn't make too much sense to have the user explicitly define the URL every single time:

COORDINATION_URL = "redis://password@hostname:6379"
bj = bigjob(COORDINATION_URL)

since it will always be the same for a given deployment. I propose to add an option to setup.py that allows to set a default URL for BigJob coordination, e.g., BIGJOB_DEFAULT_COORDINATION=redis://quary... easy_install BigJob, similar to what I do with the connection setting in bigjob-server: https://github.com/oweidner/bigjob-server/wiki/Installation.

Subsequently, the user can leave the coordination_url in the BigJob constructor blank and/or use a predefined variable, i.e.,

bj = bigjob()

or

bj = bigjob(bigjob.DEFAULT_COORD)

One could think about the same for the pilot-job LRMS, e.g.,

bj.start_pilot_job(bigjob.DEFAULT_RM,  (instead of having to remember e.g., gram://lonestar...whatever)

I think being able to define optional default options for machine-specific deployments would make BigJob more user-friendly.

pradeepmantha commented 12 years ago

I think specifying COORDINATION_URL as environment variable will help to adjust coordination system dynamically. Since this will avoid re-installing of BigJob, if COORDINATION URL need to be changed.

oleweidner commented 11 years ago

config file / environment variable support for redis connection parameters would still be useful, IMHO. Maybe it would also prevent people from putting plain-text passwords in their code...

oleweidner commented 10 years ago

Don't think that has any relevance anymore.