search

propershark / shark

An event publisher for realtime transit information.
3 stars 0 forks source link

Modular Object Storage #2

Closed faultyserver closed 8 years ago

faultyserver commented 8 years ago

CityBus is a fairly small system, with less than about 30 vehicles servicing fewer than 1000 stops, which makes storing objects in memory pretty easy. However, many transit agencies operate hundreds of vehicles across thousands of stops, if not more, which could easily put storage abilities in danger.

The solution to this is to modularize object storage into a set of adapters. The default adapter will store objects in memory, but other adapters such as Redis and SQL can persist objects in different locations to reduce overhead.

With this solution, object managers will store keys instead of full objects, and will retrieve and write the objects through the storage adapter when they are needed. Since Ruby processes are naturally single-threaded and only one process will be accessing each datum, thread safety is not yet a real concern.

To select which storage option to use, there should be configuration option that specifies which storage adapter to use:

storage_adapter: memory[default] | redis | mysql | ...

This configuration should be applied at the agency level, meaning all managers for a given agency will use the same adapter.

faultyserver commented 8 years ago

This issue is being implemented in #6.

faultyserver commented 8 years ago

Note that the configuration options are not yet implemented. However, this is a secondary issue and will be recreated as it's own issue when other adapters are implemented/it becomes relatively important.