composite-primary-keys / composite_primary_keys

Composite Primary Keys support for Active Record
1.03k stars 350 forks source link

Rails 7.1 compatibility #601

Open marcoroth opened 1 year ago

marcoroth commented 1 year ago

Hey there ๐Ÿ‘‹๐Ÿผ

First of all, thanks for the long-running support and continued maintenance of the composite_primary_keys gem! ๐Ÿ™๐Ÿผ

While following Rails Edge in one of my apps I noticed that the pull request https://github.com/rails/rails/pull/48241 changed the method signature for ConnectionAdapters#sql_for_insert from three to four arguments and Persitance#_insert_record from one to two arguments.

This caused applications using composite_primary_keys fail to boot and crash with:

ArgumentError: wrong number of arguments (given 4, expected 3) 
lib/composite_primary_keys/connection_adapters/postgresql/database_statements.rb:5:in 'sql_for_insert'`
Full Backtrace
bundle exec rails db:setup

Created database '[project]_test'
rake aborted!
ArgumentError: wrong number of arguments (given 4, expected 3)
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/composite_primary_keys-1ffb4340bc01/lib/composite_primary_keys/connection_adapters/postgresql/database_statements.rb:5:in `sql_for_insert'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:152:in `exec_insert'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/connection_adapters/postgresql/database_statements.rb:96:in `exec_insert'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/composite_primary_keys-1ffb4340bc01/lib/composite_primary_keys/connection_adapters/abstract/database_statements.rb:6:in `insert'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb:25:in `insert'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/internal_metadata.rb:123:in `create_entry'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/internal_metadata.rb:106:in `update_or_create_entry'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/internal_metadata.rb:68:in `create_table_and_set_flags'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/schema.rb:62:in `define'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/schema.rb:50:in `define'
/home/runner/work/[project]/[project]/db/schema.rb:13:in `'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:358:in `load'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:358:in `load_schema'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:453:in `block (2 levels) in load_schema_current'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:501:in `block in with_temporary_connection'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:518:in `with_temporary_pool'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:500:in `with_temporary_connection'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:452:in `block in load_schema_current'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:562:in `block (2 levels) in each_current_configuration'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:559:in `each'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:559:in `block in each_current_configuration'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:558:in `each'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:558:in `each_current_configuration'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:451:in `load_schema_current'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/railties/databases.rake:476:in `block (3 levels) in '
:88:in `require'
:88:in `require'
-e:1:in `
' Tasks: TOP => db:setup => db:schema:load (See full trace by running task with --trace) Error: Process completed with exit code 1.

And

ArgumentError: wrong number of arguments (given 1, expected 2)
rails-99f8b049f28f/activerecord/lib/active_record/persistence.rb:564:in `_insert_record'`
lib/composite_primary_keys/persistence.rb:77:in `_create_record'
Full Backtrace
bundle exec rails db:setup

Created database '[project]_test'
Running via Spring preloader in process 2631
rake aborted!
ArgumentError: wrong number of arguments (given 1, expected 2)
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/persistence.rb:564:in `_insert_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/composite_primary_keys-2a117c40935c/lib/composite_primary_keys/persistence.rb:77:in `_create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/counter_cache.rb:187:in `_create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/locking/optimistic.rb:84:in `_create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/encryption/encryptable_record.rb:174:in `_create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/attribute_methods/dirty.rb:236:in `_create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/callbacks.rb:445:in `block in _create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:110:in `run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:949:in `_run_create_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/callbacks.rb:445:in `_create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/timestamp.rb:114:in `_create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/persistence.rb:1212:in `create_or_update'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/callbacks.rb:441:in `block in create_or_update'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/autosave_association.rb:379:in `around_save_collection_association'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:130:in `block in run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:141:in `run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:949:in `_run_save_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/callbacks.rb:441:in `create_or_update'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/timestamp.rb:125:in `create_or_update'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/persistence.rb:712:in `save'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/validations.rb:49:in `save'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/transactions.rb:309:in `block in save'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/transactions.rb:365:in `block in with_transaction_returning_status'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:340:in `transaction'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/transactions.rb:361:in `with_transaction_returning_status'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/transactions.rb:309:in `save'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/suppressor.rb:52:in `save'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/persistence.rb:38:in `create'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:899:in `_create'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:103:in `block in create'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:914:in `_scoping'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:452:in `scoping'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:103:in `create'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:216:in `block in create_or_find_by'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb:487:in `block in within_new_transaction'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb:484:in `within_new_transaction'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:342:in `transaction'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/transactions.rb:212:in `transaction'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation/delegation.rb:122:in `public_send'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation/delegation.rb:122:in `block in method_missing'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:914:in `_scoping'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:452:in `scoping'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation/delegation.rb:122:in `method_missing'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:216:in `create_or_find_by'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:176:in `find_or_create_by'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/querying.rb:23:in `find_or_create_by'
/home/runner/work/[project]/[project]/db/seeds.rb:22:in `block in '
/home/runner/work/[project]/[project]/db/seeds.rb:21:in `each'
/home/runner/work/[project]/[project]/db/seeds.rb:21:in `'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/railties/lib/rails/engine.rb:556:in `load'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/railties/lib/rails/engine.rb:556:in `block in load_seed'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/execution_wrapper.rb:92:in `wrap'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/railties/lib/rails/engine.rb:642:in `block (2 levels) in '
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:130:in `instance_exec'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:130:in `block in run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:141:in `run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/railties/lib/rails/engine.rb:556:in `load_seed'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/tasks/database_tasks.rb:468:in `load_seed'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/railties/databases.rake:405:in `block (2 levels) in '
:85:in `require'
:85:in `require'
-e:1:in `
' Tasks: TOP => db:setup => db:seed (See full trace by running task with --trace) Error: Process completed with exit code 1.

This pull requests updates the methods signature for sql_for_insert in composite_primary_keys so it could take either 3 or 4 arguments and the call to _insert_record from one to two inside _create_record.

I'm also aware that Rails 7.1 is going to get some kind of support for composite primary keys, but I guess it would still make sense for composite_primary_keys to be Rails 7.1+ compatible to provide a smoother upgrade/migration path.

I'm not sure if this is the proper and correct way to fix this. At least it's making my apps test pass and lets the app boot again. So I think this pull request is at least a starting point for making composite_primary_keys Rails 7.1 compatible.

cfis commented 1 year ago

Thanks for the PR. Looks like we also need to implement the other changes though including using returning parameter?

marcoroth commented 1 year ago

If I understand the versioning right we can "break" things in a new major version since every version is just designed to work with one version of ActiveRecord, right?

So a potential 15.x release doesn't have to be compatible with ActiveRecord 7.0

cfis commented 1 year ago

Yes, that is correct. Are there big differences between 7.0 and 7.1?

Note the tests failed with this change.

machty commented 1 year ago

Hello,

I'm seeing the following in Rails 7.1 beta notes:

Support for composite primary keys in Active Record

Shopify improved the performance of common queries against their largest tables by 5-6x and reduced the number of slow queries by 80% by switching to composite primary keys. The trade-off is that inserts can become significantly slower, but for very large tables that see many more reads than writes, it can be a dramatic improvement. This work has been extracted into full support for composite primary keys in Active Record.

What does this mean for this gem? Does Rails 7.1 obsolete this gem?

cfis commented 1 year ago

Good question - don't know. Hopefully.

igor-alexandrov commented 1 year ago

I am not sure about performance of composite_primary_keys VS composite primary keys in AR 7.1 or any other aspects, but from it seems like with Rails 7.1 this gems becomes obsolete. The only difference is that in Rails 7.1 you need to use self.primary_key = [] instead of self.primary_keys = [] with this gem.

igor-alexandrov commented 1 year ago

More about composite keys support here: https://guides.rubyonrails.org/active_record_composite_primary_keys.html

cfis commented 1 year ago

It seems from #608 there isn't any reason to update CPK for Rails 7.1. Thoughts?

marcoroth commented 1 year ago

I think it might make sense to support 7.1 so you have an easier upgrade path. So that you can upgrade to Rails 7.1 without also having to migrate to the new built-in composite primary keys at the same time.

cfis commented 1 year ago

Maybe, but it sounded from the other issue the upgrade was pretty easy. But I haven't tried it myself.

codeodor commented 1 year ago

When I get around to it, I'll see if I can upgrade my app by changing CPK to just add the code:

def self.primary_keys = (array)
  self.primary_key = array
end

Which should theoretically work?

cfis commented 1 year ago

And the :foreign_key => [a,b] to :query_constraints it looked. ย  ย  On 2023-10-11 13:41, Sammy Larbi @.***> wrote:

When I get around to it, I'll see if I can upgrade my app by changing CPK to just add the code:

def self.primary_keys = (array) self.primary_key = array end

Which should theoretically work?

โ€”

Reply to this email directly, view it on GitHub (https://github.com/composite-primary-keys/composite_primary_keys/pull/601#issuecomment-1758505894), or unsubscribe (https://github.com/notifications/unsubscribe-auth/AAAERRRQDJEZPQZVYPCGG53X63765ANCNFSM6AAAAAA2OWQUDM).

You are receiving this because you commented.Message ID: @.***>

[ { @.": "http://schema.org", @.": "EmailMessage", "potentialAction": { @.": "ViewAction", "target": "https://github.com/composite-primary-keys/composite_primary_keys/pull/601#issuecomment-1758505894", "url": "https://github.com/composite-primary-keys/composite_primary_keys/pull/601#issuecomment-1758505894", "name": "View Pull Request" }, "description": "View this Pull Request on GitHub", "publisher": { @.": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

jarl-dk commented 1 year ago

FYI: I have opened a bug report on rails regarding composite primary keys: https://github.com/rails/rails/issues/49597

marcocarvalho commented 4 months ago

Hello @marcoroth any plans to continue this implementation?

marcoroth commented 3 months ago

@marcocarvalho I migrated to the built-in Rails solution in the meantime. I also realized that my approach is kinda flawed and somehow only just worked to make my tests pass.

I think this would need another more serious take to make it work with Rails 7.1.