With dispatcher - Githubissues

eligosource / eventsourced

A library for building reliable, scalable and distributed event-sourced applications in Scala

Apache License 2.0

828 stars 98 forks source link

With dispatcher #37

Closed rocketraman closed 11 years ago

rocketraman commented 11 years ago

This adds a dispatcher configuration for channels (to be configured by callers) as well as a dispatcher for the Journal actor. It bounds the journal actor mailbox by default, in case I/O cannot keep up with the events.

rocketraman commented 11 years ago

The branch is on top of wip-serializer, just look at the last two commits...

rocketraman commented 11 years ago

Nope, still not completely fixed... the queue in org.eligosource.eventsourced.core.ReliableChannelDeliverer still grows without bound.

krasserm commented 11 years ago

Hi Raman, thanks for your contribution. I made some additions/changes and pushed to wip-dispatcher

Child actors of ReliableChannel are configured to use the same dispatcher that is used for ReliableChannel. This should fix your issues with ReliableChannelDeliverer.
Journal factories now have a dispatcher name parameter i.e. you now can create journals with a custom dispatcher like LeveldbJournal(dir, dispatcherName = Some("my-dispatcher")). I dropped the the default journal disptacher configuration from reference.conf

Please let me know if this works for you. There are still some options to improve memory consumption of ReliableChannel, I'll work on that later.

rocketraman commented 11 years ago

Thanks for the updates! I realized that LevelDB is actually quite slow to deal with large values -- it keeps trying to reorganize the journal, and writing large values to it kills it's performance. What I'm experimenting with now is writing the "blob" values to the filesystem, and keeping the meta-data/event for subsequent actor processing in the journal via eventsourced. Working out a few kinds but Initial investigation into this approach is good.