Any plans to support weak/hybrid schemas?

[caws]

Is your feature request related to a problem? Please describe.

Versioning events can be complicated and tricky (as per described here), but weak/hybrid schemas can help.

Please correct me if I'm wrong, but as far as I can tell it seems the current event setup supports strong schemas only. That means that the following can be used to represent some event:

  class TestEvent < ActiveEventStore::Event  
    attributes :user_id, :action_type
  end

But maybe backwards compatibility can become an issue if/when the requirements are updated and the data being passed around by this event needs to be changed, especially when removing an attribute (as then we get: ArgumentError, Unknown event attributes: user_id).

There are a few ways to tackle this issue:

• Introduce a new event type and slowly migrate all consumers to be compatible with it is an option, but that would require littering the code with multiple event definitions (v1, v2, v3) that could all be slightly different and could introduce other more complex issues as explained here.
• Double publish is also an option, as in publishing both versions of the event and having the subscribers deal only with the events that they understand, but issues when replaying the events arise from this approach.

And some others, but none of which make this 'migration' easier/more transparent as we'd expect from something out of the box for Rails as far as I can humbly tell.

Among the approaches described in that book, I noticed that the weak schema may likely be the 'more transparent' one, as it allows one to have a well defined event type (whose name would likely not change as attributes are added/removed) while moving the responsibility of deserializing it into a 'mapper' class that handles the presence/absence of attributes in a more graceful manner.

Describe the solution you'd like

Perhaps a possible solution would be to have a Schema parent class from which other schemas could inherit.

This Schema base class could implement an .attribute method that receives both the attribute name (like what happens already) and a default value as a fallback (nil if not present).

This could allow us to have Events and Schemas defined as follows:

class TestEventSchema < ActiveEventStore::Schema
  attribute :user_id, nil
  attribute :action_type, 'some_default_value_here_aye?'
end

class TestEvent < ActiveEventStore::Event  
   with_schema TestEventSchema
end

The attributes could be processed through the schema just before invoking validate_attributes! if a schema was present for the event.

The rest would stay as is and minimal changes would be introduced to the current setup.

I see a few pros:

• No new events created to represent slightly the same thing, we'd use the same event class already in place.
• We can add/remove attributes at will to the weak schema without risking a failure
• If a given attribute is not present when a consumer tries to consume an event we can rely on a default fallback (or raise an error instead?)

And a few cons:

• Renaming is an issue
• Cannot go back to using strong schemas (unless you replicate the same list of attributes present in the schema)
• Could probably force checks to be added to the consumer to ensure a given value is in place (or raise error instead?)

Describe alternatives you've considered

Adding new events, double publish, etc.

Additional context

I have worked on a POC branch that adds support to weak schemas and another one that expands on that and adds support to hybrid schemas.

I can open a PR if that is a direction the core team would like to take, of course. If not, I'd like to ask how you currently tackle the issue of migrating events while maintaining their immutability. :thinking:

Thanks!!

[palkan]

I don't have such plans, because we haven't hit this problem yet; but I believe, we'll eventually do; so, we need to solve this.

Thanks for the examples and links; I got the following idea—add ignore_attributes and alias_attribute DSL methods. That sounds to me like a pretty Rails way of dealing with that (since we already have ignore_columns and alias_attribute for AR for the exactly same reasons). WDYT?

[caws]

I don't have such plans, because we haven't hit this problem yet; but I believe, we'll eventually do; so, we need to solve this.

Thanks for the examples and links; I got the following idea—add ignore_attributes and alias_attribute DSL methods. That sounds to me like a pretty Rails way of dealing with that (since we already have ignore_columns and alias_attribute for AR for the exactly same reasons). WDYT?

I see, I suggested the schema approach because then we could move that responsibility into a separate class, however that would add complexity to the approach. Indeed what you suggested sounds more railsy. :thinking:

What about default/fallback values for attributes? Separate issue?

[palkan]

What about default/fallback values for attributes? Separate issue?

Yeah, I think so. That's an open question at which moment we should populate the defaults 🤔 So, there is something to discuss.