Update ViewSchema.create_view_tables method to execute alter migrations

[smilyalexey]

Currently I faced with an issue in my test environment that all alter migrations are ignored there. I found that when I call Sequent::Test::DatabaseHelpers.maintain_test_database_schema it calls only 2 methods: create_sequent_schema_if_not_exists and create_view_tables. And there is nothing about calling alter migrations.

I couldn't find any info about that neither in examples or docs. So I decided to update create_view_tables method. Let me know if it's possible to handle in different way or if you prefer to handle it in different way.

[lvonk]

Closing as this is missing in the docs. Perhaps we will change this in the future but for now it works as commented.

[smilyalexey]

@lvonk thanks for the info. I think it's important to add this in the docs, otherwise the main flow is not obvious. Working on some very simple project - looks awesome, but once you start using sequent in production it become a headache.

I like the idea, and framework in general, but lucking of documentation and having the issues in some very obvious things make almost impossible to use sequent in production.

If you have any group or small community - I'll be happy to join to that.

[lvonk]

If you have any group or small community - I'll be happy to join to that.

Well this is the place to contribute, please keep reporting the issues in the very obvious things so we can fix them.

We have been using sequent in production for many years in multiple applications with billions of events.

So can you share what gives you the headache?

[smilyalexey]

I beleive its usable in production and I understand the reason why you publish it is because you had a big succesfull experience to use that in your products. But the biggest headache is - pure documentation with using sequent + rails and low community.

I had to use a bit more monkey patches that I initially mentioned here - https://github.com/zilverline/sequent/issues/375

Migrations in general looks buggy to me:

• When I run online migrations it usually doesn't work in the first attempt and I had to run it in the second time that usually succeed. In general it doesn't give you confidence with running your migrations.
• Once I decided to re-fill view tables from scratch (because it was the first version of schema and instead of creating the second version I decided to re-create tables and re-stream the events and commands). At the end I found a few missed records in view tables. I have feeling it might be related to the issue with failed first attempt to run online migrations. I didn't dig into that too deep as app wasn't fully enabled yet.
• I saw some sequent tables in public schema when I tried to create sequent schemas and tables. I fixed it via specifying SEQUENT_MIGRATION_SCHEMAS="sequent_schema, view_schema" ...

PS in database.yml it's like this - schema_search_path: "<%= ENV['SEQUENT_MIGRATION_SCHEMAS'] || 'public, sequent_schema, view_schema' %>"

• also it's not clear what's the best way to add a new field and run migrations. For example I wanted to add domain in read models and found that there is no such info in event. So I couldn't find any examples how I can make it work, however versions for event's is something obvious. I found only Upcasting that supposed to help to solve this problem, but since I already had issues with running the migrations I decided to update it manually for read models. I know that it's wrong, but at the same time I don't know how to handle it in right way.

[lvonk]

But the biggest headache is - pure documentation with using sequent + rails and low community.

Yes the event sourcing community in ruby land is not very big. This year we really started with the Sequent Rails integration and are running this for a few months in production ourselves as well.

When I run online migrations it usually doesn't work in the first attempt and I had to run it in the second time that usually succeed. In general it doesn't give you confidence with running your migrations.

I can imagine this does not give you confidence. This is not how is should work and not how we (ever) experienced this in the last decade we use sequent. Things to look for if the migrations don't run: is the version updated in the migrations file? Are the"migrations" listed in the migrations file under the correct version? Do you get an error message?

Once I decided to re-fill view tables from scratch (because it was the first version of schema and instead of creating the second version I decided to re-create tables and re-stream the events and commands). At the end I found a few missed records in view tables

Again this indeed does not give confidence. As long as the events are in the event store recreating the projections should work. If for some reason replaying fails, the process exits and the app stays on the current version.

PS in database.yml it's like this - schema_search_path: "<%= ENV['SEQUENT_MIGRATION_SCHEMAS'] || 'public, sequent_schema, view_schema' %>"

This is the way to fix sequent tables winding up in your public schema, also ensure you follow the steps decribed in Rails and Sequent guide to not run the rails db:migrate directly.

also it's not clear what's the best way to add a new field and run migrations

What field? You can always add attributes in your event. You can use upcasting as a mean to give it a default for value for events that are persisted but do not have this value. You can of course also choose to create an entirely new event. But that really depends on your domain.

Don't hesitate to reach out and ask questions via the issues. Hope this helps.

[smilyalexey]

This year we really started with the Sequent Rails integration and are running this for a few months in production ourselves as well.

If you can review and update your docs and provide a bit more rails examples - that really will help with that. Right now it's almost impossible to find any samples on ruby on rails or even just sequent (only those that you already provided) + even when you asked chatgpt about sequent - it gave me an examples that looked like the mix of sequent and rails_event_store 😄

I'll create the list of issues for each of my question and will provide more details there. Thanks a lot for your reply.