Stream prefix not working?

[drozzy]

I'm using commanded_extreme_adapter ~> 0.4 and for some reason configuring stream_prefix doesn't work:

config :commanded_extreme_adapter,
  serializer: Commanded.Serialization.JsonSerializer,
  stream_prefix: "blah"

Still all my streams are created with prefix "commanded".

[drozzy]

Huh... strange, it started working after I deleted all my deps, _build and mix.lock and recompiled everything.

I wonder why it would not work before?

[drozzy]

Ok, I just changed it again and just did mix compile and nothing happened.

I assume this is some sort of macro magic? (I'm not an expert in that stuff)

[slashdotdash]

Commanded uses the stream_prefix for two reasons:

1. To allow multiple Commanded apps to use a single Event Store instance.
2. For event handlers to subscribe to a category stream (using the configured stream prefix) so that only application specific events are received. This means that it can ignore any Event Store events that would otherwise be present if the $all stream was used.

The Commanded Extreme adapter won't work if you configure an empty stream prefix.

The stream_prefix configuration is read at compile time, which means that if you change it you must force a recompile of this dependency (mix compile --force commanded_extreme_adapter). It might be more user friendly to lookup the value at runtime, especially as this force recompile isn't mentioned in the documentation.

[drozzy]

Thanks. A few points:

1. Why can't multiple commanded apps work with the same prefix? It will just work because of event number consistency mechanism.
2. I'm not sure I follow this... maybe we can specify which streams Commanded should follow explicitly? That way there is not need to guess.

Yes, I'm all for dynamic lookups! :-)

[slashdotdash]

1. Why can't multiple commanded apps work with the same prefix?

They wouldn't be able to deserialize each other's domain events, unless you shared the Elixir structs, which would make them tightly coupled, not autonomous.

2. I'm not sure I follow this... maybe we can specify which streams Commanded should follow explicitly? That way there is not need to guess.

Usually you have a stream per aggregate instance, therefore it's not possible to know in advance which streams to subscribe to (e.g. user-123, user-456). That's why we use Event Store's built-in support for building category projections using the stream-prefix setting. A projection also provides the event ordering/numbering guarantee to support subscription pointers.

[drozzy]

Thanks for replies. Here are my thoughts:

1. I see nothing wrong with sharing Elixir structs. I do this presently - I just write exactly the same structs on different nodes. Sharing events is perfectly OK imo. Alternatively, I would be happy to write serialize/deserialize methods for any other events that don't automatically work.

2. What about categories? You can subscribe to stream $ce-user and it will give you all those events for all those streams: user-123, and user-456. I mean if you leave it up to the user to specify this (e.g. which category corresponds to which aggregate) or use defaults ("User" aggregate corresponds to $ce-user stream) - both will work I think.

[drozzy]

See byCategory built-in projection here: https://eventstore.org/docs/projections/system-projections/

By default the $by_category projection will link existing events from a stream id with a name such as account-1 to a stream called $ce-account.

[slashdotdash]

The category projection is what's already being used. The stream_prefix is the category, allowing Commanded to subscribe to all events, regardless of which aggregate type/instance created them. It also provides consistent global ordering to events.

Per aggregate subscriptions are possible, but require changes to Commanded and this adapter. Happy to accept a pull request.

[drozzy]

I don't trust myself to implement it :-(

But if I did, I would get rid of stream_prefix completely. Anyways, for now I'll just use a stream prefix of "aggregate-" or something like that. Adds a bit of meaning to my stream names, I guess. Thanks!

[drozzy]

Just tried changing stream_prefix and compiling with:

mix compile --force commanded_extreme_adapter

and it seems like it's still using the old one.

But removing _build directory to recompile everything worked.

[slashdotdash]

Compile-time constants are a headache. I will replace them with runtime lookup to prevent the annoyance of having to mess around with force recompile and/or removing the _build directory.

[drozzy]

@slashdotdash Ok, my hack of altering $byCategory system projection doesn't work.

The problem is that some streams have UUIDs in them, e.g.:

aggregate-model-d8f93f0d-73c9-4995-b60b-3ca8e06e3684

and setting this to "last" hyphen actually makes the category "aggregate-model-d8f93f0d-73c9-4995-b60b" and not "aggregate-model" as desired.

So I think ES semantics of having the first dash denote the category is correct.

Is there any way to change commanded's delimiter from a "dash" to something else? E.g. ":"?

[slashdotdash]

@drozzy Let me take a look at how best to configure Commanded to support your scenario.

I'm curently thinking if the stream prefix config is made optional, but add it to the event handler and process manager use options. That should allow you to use the default Event Store category projections by prefixing your aggregate's identity with their type (e.g. user-) and defining the same prefix in an event handler to handle all user events.

[drozzy]

Ok, thanks (I'm not sure I follow what you mean, but I think you want to re-use ES prefix?).

There is a "hack" I could do in ES, and that is to replace their delimiter by another symbol, like a colon ":". But I'll have to change a lot of code, so I'd rather hold off on this for now.

[drozzy]

@slashdotdash Oh no wait, changing the symbol to : won't work, because commanded will persist with - anyways (I'm assuming).

[drozzy]

Hm.. I still don't understand the need for prefix. Looking at the EventStore behaviour: https://github.com/commanded/commanded/blob/master/lib/commanded/event_store/event_store.ex

is there some specific aspect of it that requires this? I don't think having multiple commanded apps on the same node makes sense. Having them on distributed nodes will work. After all, we have optimistic concurrency.

Hm.. let me play around with this.

[drozzy]

Closing this for now.