Allow specifying index

[sfcgeorge]

As discussed partitioning is a great way to scale a big table like events. https://github.com/orgs/RailsEventStore/discussions/1123 https://github.com/orgs/RailsEventStore/discussions/1163

Why use partitions (tangent)

• A partitioned table appears just like any other table, no code or schema changes needed (mostly).
• Partitioning can be done just in production where it's actually needed. Rails schema.rb doesn't support partitioned tables so it's best not to partition in development.
• Old partitions (if partitioned by created_at) can be backed up and dropped to save space. We keep 13 months for debugging but drop any older partitions.
• Dropping partitions is instant whereas simply deleting millions of old rows would be very slow and could take your site down.

How to partition (tangent)

To partition a table it is necessary to have a unique index on the partitioning column. We wanted to partition monthly so this is what we did:

class UpdateEventsIndexForPartitioning < ActiveRecord::Migration[7.0]
  disable_ddl_transaction!

  def change
    # A unique index on the partitioning column is needed to partition.
    # In this case created_at may not be unique so we include event_id too.
    add_index :event_store_events, [:created_at, :event_id],
              unique: true, algorithm: :concurrently
    # These indexes are no longer needed as the columns are included in the new index.
    remove_index :event_store_events, :event_id,
                 unique: true, algorithm: :concurrently
    remove_index :event_store_events, :created_at, algorithm: :concurrently
  end

end

Then we used PGSlice to run partitioning commands against production as it makes it really easy, but it just generates and runs SQL commands which could be written manually instead.

pgslice prep event_store_events created_at month
pgslice add_partitions event_store_events --intermediate --past 50 --future 1
pgslice fill event_store_events
pgslice analyze event_store_events
pgslice swap event_store_events
pgslice fill event_store_events --swapped

We've also set up some CRON rake tasks that wrap PGSlice to create new partitions and delete old ones.

Issue

RES doesn't allow specifying a different index for the RubyEventStore::ActiveRecord::Event model. For reads this is fine, but for writes you get an error No unique index found for #{name_or_columns}. Trace:

activerecord (7.1.3) lib/active_record/insert_all.rb in find_unique_index_for at line 161
activerecord (7.1.3) lib/active_record/insert_all.rb in initialize at line 35
activerecord (7.1.3) lib/active_record/persistence.rb in new at line 243
activerecord (7.1.3) lib/active_record/persistence.rb in insert_all! at line 243
ruby_event_store-active_record (2.14.0) lib/ruby_event_store/active_record/event_repository.rb in block in append_to_stream at line 27
ruby_event_store-active_record (2.14.0) lib/ruby_event_store/active_record/event_repository.rb in block in add_to_stream at line 104
activerecord (7.1.3) lib/active_record/connection_adapters/abstract/transaction.rb in block in within_new_transaction at line 535
activesupport (7.1.3) lib/active_support/concurrency/null_lock.rb in synchronize at line 9
activerecord (7.1.3) lib/active_record/connection_adapters/abstract/transaction.rb in within_new_transaction at line 532
activerecord (7.1.3) lib/active_record/connection_adapters/abstract/database_statements.rb in transaction at line 344
activerecord (7.1.3) lib/active_record/transactions.rb in transaction at line 212
ruby_event_store-active_record (2.14.0) lib/ruby_event_store/active_record/event_repository.rb in start_transaction at line 160
ruby_event_store-active_record (2.14.0) lib/ruby_event_store/active_record/event_repository.rb in add_to_stream at line 103
ruby_event_store-active_record (2.14.0) lib/ruby_event_store/active_record/event_repository.rb in append_to_stream at line 27
ruby_event_store (2.14.0) lib/ruby_event_store/instrumented_repository.rb in block in append_to_stream at line 12
activesupport (7.1.3) lib/active_support/notifications.rb in instrument at line 208
ruby_event_store (2.14.0) lib/ruby_event_store/instrumented_repository.rb in append_to_stream at line 11
ruby_event_store (2.14.0) lib/ruby_event_store/client.rb in append_records_to_stream at line 359
ruby_event_store (2.14.0) lib/ruby_event_store/client.rb in publish at line 35

Setting the AR primary key to the new compound unique index would fix it, but there's no RES config to change the primary key for the event model.

Temporary fix

We can monkeypatch RES in an initializer but it's particularly gross because RES makes the Event class constant private so we have to get a bit hacky:

RubyEventStore::ActiveRecord.const_get('Event').primary_key = [:created_at, :event_id]

Request

I guess either Event could be made public, or there could be a new config for setting the primary key. Maybe there's a more comprehensive solution.

[swistak35]

Writing out of my head, so I may be incorrect, but:

Rails Event Store's configuration allows to replace the repository. The repository, on the other hand, allows you to replace the Event model (the one you need to override). So if I'm understanding your issues correctly, you should be able to do something like this:

module MyEventStore
  class Event < ActiveRecord::Base
    self.table_name = '...'
    self.primary_key = [:created_at, :event_id]
  end

  class EventInStream < ActiveRecord::Base
    self.table_name = '...'
  end

  class ModelFactory
    def call
      [Event, EventInStream]
    end
  end
end

And then, in the place where you configure event store, you can ps

event_store = RailsEventStore::Client.new(
  repository: RailsEventStoreActiveRecord::EventRepository(serializer: RubyEventStore::Serializers::YAML, event_factory: MyEventStore::ModelFactory)
)

For the default implementation which MyEventStore::ModelFactory has to follow, look at this: https://github.com/RailsEventStore/rails_event_store/blob/master/ruby_event_store-active_record/lib/ruby_event_store/active_record/with_default_models.rb

Let me know if that is a good direction or if I should look more into your issue.

PS. What you did sounds like a great blogpost, I don't recall anyone already describing partitioning with RES in such detail.

[sfcgeorge]

Thank you very much! It's been years since I set up event store so I forgot how composable it was.

I did basically what you said and it seems to work perfectly. I didn't need to override event in stream so I managed to reuse that:

class EventStore
  class EolaEvent < ::ActiveRecord::Base
    self.primary_key = [:created_at, :event_id]
    self.table_name = "event_store_events"
  end

  class EolaFactory < RubyEventStore::ActiveRecord::WithDefaultModels
    def call
      _, event_in_stream = super
      [EolaEvent, event_in_stream]
    end
  end

  # ...

  def self.setup
    Rails.configuration.event_store = RailsEventStore::Client.new(
      repository:   RailsEventStoreActiveRecord::EventRepository.new(
        model_factory: EolaFactory.new
        # ...
      ),
      # ...
    )
  end
end

I am very good at procrastinating writing blog posts, but thank you, I'll try to write one!

I think I'll close this issue as there is an official way to override the whole Event class. As other stores like Mongo are supported which may not even have primary keys it wouldn't make sense to add a specific config for that.

RES could add built-in support for partitioning, but I think it would be hard to make it generic enough.