search

meetings / gearsloth

Gearman job persistence and delayed execution system
MIT License
2 stars 0 forks source link

Isolated storage system support (aka db specific ejector functions) #65

Closed amv closed 10 years ago

amv commented 10 years ago

Currently the ejector logic works so that each ejector should be connected to all of the databases in the system. This is pretty inflexible as it makes it practically impossible to use filesystem based storage schemes without a single point of failure (proper Storage Area Network systems have proven to be pretty expensive to maintain).

We have discussed that the system specifications would be changed so that instead of publishing a single "eject" function, the ejectors publish a separate eject function for each database system that they are connected to. The runner who queries a specific database system is then responsible for amending the task description so that it contains the literal function name that should be used to eject the task from the database it exists in.

This means the adapter interface needs to be extended to provide a way for the ejector to find out which functions it should publish to the gearmand cluster.

This alteration has the added benefit of easing the development of custom controllers as the controllers do not need to be aware of the system specific ejector function names. They just have a standard task attribute that specifies the function name to use when ejecting the task.

amv commented 10 years ago

This should be solved in the fs-adapter branch