kaazing / gateway

Kaazing Gateway
Apache License 2.0
141 stars 84 forks source link

Add.hazelcast.topics #862

Closed Anisotrop closed 7 years ago

Anisotrop commented 7 years ago

First implementation of topics interface defined in Hazelcast. Using 2 executor services:

cmebarrow commented 7 years ago

@Anisotrop I am uneasy about the use of concurrent hashmap and its entryset method in MemoryTopic. It will create garbage and slow up the system. publishing is going to be a frequent event so hot code. I think there are two ways we can go here:

  1. Call the listeners directly from the publish implementation, i.e. do not use a separate thread at all. Use a concurrent data structure to hold the list of listeners (probably CopyOnWriteArrayList). I think this might be the best way to go because it's easy, garbage free, and when we use the feature in the MQ driver the listeners will be extremely lightweight (they were just post a command onto an Agrona queue which is processed by the queue reader thread). Obviously you would have to forbid calls to publish, add or remove listener from within a llistener, but as I said already in another comment I think that would be just fine.
  2. Post add listener, remove listener and publish as commands onto an Agrona queue which is processed by a worker thread (could have just one thread per MemoryCollectionsFactory instance, it can handle all topics). With this approach everything is single threaded and ordering is guaranteed correct. The thread lifecycle would be managed by the MemoryCollectionsFactory instance (so not using SchedulerProvider).

We can talk tomorrow if you want, after the dev meeting.