search

ATIX-AG / foreman_scc_manager

Foreman plugin to sync SUSE Customer Center products and repositories into Katello
GNU General Public License v3.0
15 stars 11 forks source link

DB migration fails with Foreman 3.3 / Katello 4.5 because of already existing permissions #104

Closed dgoetz closed 1 year ago

dgoetz commented 2 years ago

After installing the plugin, when running the foreman-installer I got an error that foreman-rake db:migrate failed. To reproduce it and provide some clean output I build a new machine.

As you can see I prepared the repos, installed and run the installer. Afterwards I verified the permissions are not present. Then I installed the plugin and another verification already shows the permissions. When I run the migrations now it fails. Deleting the migration seems fine as migrate and seed run fine afterwards as the installer does and from looking into the Roles "SCCViewer" and "SCCManager" I would say it is also fine. So I have no idea why it fails now as it seems like the migration is in for a while and I also do not know how the permissions are created during plugin installation without any scripts or commands executed.

# dnf localinstall https://yum.theforeman.org/releases/3.3/el8/x86_64/foreman-release.rpm -y
# dnf localinstall https://yum.theforeman.org/katello/4.5/katello/el8/x86_64/katello-repos-latest.rpm -y
# dnf localinstall https://yum.puppet.com/puppet7-release-el-8.noarch.rpm -y
# dnf config-manager --set-enabled powertools
# dnf module enable katello:el8 pulpcore:el8 -y
# dnf install foreman-installer-katello -y
# foreman-installer --scenario katello --skip-checks-i-know-better --disable-system-checks
# foreman-rake console
Loading production environment (Rails 6.0.4.7)
irb(main):001:0> Permission.where(resource_type: 'SccAccount')
=> #<ActiveRecord::Relation []>
irb(main):002:0> Permission.where(resource_type: 'SccProduct')
# dnf install rubygem-foreman_scc_manager -y
# foreman-rake console
Loading production environment (Rails 6.0.4.7)
irb(main):001:0> Permission.where(resource_type: 'SccAccount')
=> #<ActiveRecord::Relation [#<Permission id: 224, name: "view_scc_accounts", resource_type: "SccAccount", created_at: "2022-08-18 12:25:23", updated_at: "2022-08-18 12:25:23">, #<Permission id: 225, name: "use_scc_accounts", resource_type: "SccAccount", created_at: "2022-08-18 12:25:23", updated_at: "2022-08-18 12:25:23">, #<Permission id: 226, name: "new_scc_accounts", resource_type: "SccAccount", created_at: "2022-08-18 12:25:23", updated_at: "2022-08-18 12:25:23">, #<Permission id: 227, name: "edit_scc_accounts", resource_type: "SccAccount", created_at: "2022-08-18 12:25:23", updated_at: "2022-08-18 12:25:23">, #<Permission id: 228, name: "delete_scc_accounts", resource_type: "SccAccount", created_at: "2022-08-18 12:25:23", updated_at: "2022-08-18 12:25:23">, #<Permission id: 229, name: "sync_scc_accounts", resource_type: "SccAccount", created_at: "2022-08-18 12:25:23", updated_at: "2022-08-18 12:25:23">, #<Permission id: 230, name: "test_connection_scc_accounts", resource_type: "SccAccount", created_at: "2022-08-18 12:25:23", updated_at: "2022-08-18 12:25:23">]>
irb(main):002:0> Permission.where(resource_type: 'SccProduct')
=> #<ActiveRecord::Relation [#<Permission id: 231, name: "view_scc_products", resource_type: "SccProduct", created_at: "2022-08-18 12:25:23", updated_at: "2022-08-18 12:25:23">, #<Permission id: 232, name: "subscribe_scc_products", resource_type: "SccProduct", created_at: "2022-08-18 12:25:23", updated_at: "2022-08-18 12:25:23">]>
# foreman-rake db:migrate
== 20170221100619 CreateSccAccounts: migrating ================================
-- create_table(:scc_accounts, {:id=>:integer})
   -> 0.0125s
== 20170221100619 CreateSccAccounts: migrated (0.0126s) =======================

== 20170227103408 CreateSccProducts: migrating ================================
-- create_table(:scc_products, {:id=>:integer})
   -> 0.0137s
== 20170227103408 CreateSccProducts: migrated (0.0138s) =======================

== 20170301131641 CreateSccRepositories: migrating ============================
-- create_table(:scc_repositories, {:id=>:integer})
   -> 0.0134s
== 20170301131641 CreateSccRepositories: migrated (0.0135s) ===================

== 20170301141330 CreateSccProductsSccRepositoriesJoinTable: migrating ========
-- create_join_table(:scc_products, :scc_repositories, {:column_options=>{:type=>:integer}})
   -> 0.0010s
== 20170301141330 CreateSccProductsSccRepositoriesJoinTable: migrated (0.0011s) 

== 20170301163451 AddProductTypeToSccProduct: migrating =======================
-- add_column(:scc_products, :product_type, :string, {:limit=>63})
   -> 0.0004s
== 20170301163451 AddProductTypeToSccProduct: migrated (0.0005s) ==============

== 20170302082912 RemoveRepositoriesFromSccProducts: migrating ================
-- remove_column(:scc_products, :repositories, :string)
   -> 0.0006s
== 20170302082912 RemoveRepositoriesFromSccProducts: migrated (0.0006s) =======

== 20170302121542 CreateSccExtendings: migrating ==============================
-- create_table(:scc_extendings, {:id=>:integer})
   -> 0.0159s
-- add_foreign_key(:scc_extendings, :scc_products, {:column=>:scc_product_id})
   -> 0.0007s
-- add_foreign_key(:scc_extendings, :scc_products, {:column=>:scc_extension_id})
   -> 0.0007s
== 20170302121542 CreateSccExtendings: migrated (0.0173s) =====================

== 20170303085304 AddOrganizationToSccAccount: migrating ======================
-- add_column(:scc_accounts, :organization_id, :integer, {:null=>true, :index=>true})
   -> 0.0005s
-- change_column(:scc_accounts, :organization_id, :integer, {:index=>true})
   -> 0.0003s
== 20170303085304 AddOrganizationToSccAccount: migrated (0.0026s) =============

== 20170303131704 AddProductIdToSccProduct: migrating =========================
-- add_column(:scc_products, :product_id, :integer, {:index=>true, :null=>true})
   -> 0.0004s
-- add_foreign_key(:scc_products, :katello_products, {:column=>:product_id, :on_delete=>:nullify})
   -> 0.0009s
== 20170303131704 AddProductIdToSccProduct: migrated (0.0013s) ================

== 20170307092057 AddSyncedToSccAccount: migrating ============================
-- add_column(:scc_accounts, :synced, :datetime, {:default=>nil})
   -> 0.0005s
== 20170307092057 AddSyncedToSccAccount: migrated (0.0005s) ===================

== 20170418132648 AddNameToSccAccount: migrating ==============================
-- add_column(:scc_accounts, :name, :string, {:limit=>255})
   -> 0.0003s
== 20170418132648 AddNameToSccAccount: migrated (0.0003s) =====================

== 20170505063726 AddSyncStatusToSccAccount: migrating ========================
-- add_column(:scc_accounts, :sync_status, :string, {})
   -> 0.0004s
== 20170505063726 AddSyncStatusToSccAccount: migrated (0.0004s) ===============

== 20180321000000 ChangeSyncStatusToSyncTask: migrating =======================
-- remove_column(:scc_accounts, :sync_status, :string)
   -> 0.0004s
-- add_column(:scc_accounts, :sync_task_id, :uuid, {:null=>true})
   -> 0.0002s
== 20180321000000 ChangeSyncStatusToSyncTask: migrated (0.0007s) ==============

== 20190417202427 AddRecurringSync: migrating =================================
-- add_column(:scc_accounts, :foreman_tasks_recurring_logic_id, :integer)
   -> 0.0003s
-- add_column(:scc_accounts, :interval, :string, {:default=>"never"})
   -> 0.0003s
-- add_column(:scc_accounts, :sync_date, :datetime)
   -> 0.0001s
-- add_foreign_key(:scc_accounts, :foreman_tasks_recurring_logics, {:name=>"scc_accounts_foreman_tasks_recurring_logic_fk", :column=>"foreman_tasks_recurring_logic_id"})
   -> 0.0008s
-- add_column(:scc_accounts, :task_group_id, :integer, {:index=>true})
   -> 0.0001s
-- add_foreign_key(:scc_accounts, :foreman_tasks_task_groups, {:column=>:task_group_id})
   -> 0.0005s
== 20190417202427 AddRecurringSync: migrated (0.0029s) ========================

== 20200520281300 FixSccPermissions: migrating ================================
== 20200520281300 FixSccPermissions: migrated (0.0060s) =======================

== 20201119084201 AddGpgKeyToSccAccount: migrating ============================
-- add_column(:scc_accounts, :katello_gpg_key_id, :integer, {:null=>true})
   -> 0.0005s
-- add_foreign_key(:scc_accounts, :katello_content_credentials, {:column=>:katello_gpg_key_id, :on_delete=>:nullify})
   -> 0.0014s
== 20201119084201 AddGpgKeyToSccAccount: migrated (0.0025s) ===================

== 20210205082733 AddSubscriptionValidToSccProductsAndRepos: migrating ========
-- add_column(:scc_products, :subscription_valid, :boolean, {:null=>true})
   -> 0.0004s
-- add_column(:scc_repositories, :subscription_valid, :boolean, {:null=>true})
   -> 0.0002s
== 20210205082733 AddSubscriptionValidToSccProductsAndRepos: migrated (0.0007s) 

== 20210210104407 AddRootRepositoryIdToSccRepository: migrating ===============
-- add_column(:scc_repositories, :katello_root_repository_id, :integer, {:index=>true, :null=>true})
   -> 0.0003s
-- add_foreign_key(:scc_repositories, :katello_root_repositories, {:column=>:katello_root_repository_id, :on_delete=>:nullify})
   -> 0.0008s
== 20210210104407 AddRootRepositoryIdToSccRepository: migrated (0.0012s) ======

== 20210224095050 ConnectKatelloRootRepositoryToSccRepository: migrating ======
== 20210224095050 ConnectKatelloRootRepositoryToSccRepository: migrated (0.0021s) 

== 20210713092440 AddPermissions: migrating ===================================
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

Validation failed: Name has already been taken
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/validations.rb:80:in `raise_validation_error'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/validations.rb:53:in `save!'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:318:in `block in save!'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:375:in `block in with_transaction_returning_status'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/database_statements.rb:278:in `transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:212:in `transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:366:in `with_transaction_returning_status'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:318:in `save!'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/suppressor.rb:48:in `save!'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/persistence.rb:55:in `create!'
/usr/share/gems/gems/foreman_scc_manager-1.8.20/db/migrate/20210713092440_add_permissions.rb:8:in `up'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:831:in `exec_migration'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:812:in `block (2 levels) in migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:811:in `block in migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:471:in `with_connection'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:810:in `migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1002:in `migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1310:in `block in execute_migration_in_transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1361:in `block in ddl_transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/database_statements.rb:280:in `block in transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/transaction.rb:280:in `block in within_new_transaction'
/usr/share/gems/gems/activesupport-6.0.4.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
/usr/share/gems/gems/activesupport-6.0.4.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
/usr/share/gems/gems/activesupport-6.0.4.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
/usr/share/gems/gems/activesupport-6.0.4.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
/usr/share/gems/gems/activesupport-6.0.4.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/transaction.rb:278:in `within_new_transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/database_statements.rb:280:in `transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:212:in `transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1361:in `ddl_transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1309:in `execute_migration_in_transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1281:in `block in migrate_without_lock'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1280:in `each'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1280:in `migrate_without_lock'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1229:in `block in migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1382:in `with_advisory_lock'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1229:in `migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1061:in `up'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1036:in `migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/tasks/database_tasks.rb:238:in `migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/railties/databases.rake:86:in `block (3 levels) in <top (required)>'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/railties/databases.rake:84:in `each'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/railties/databases.rake:84:in `block (2 levels) in <top (required)>'
/usr/share/gems/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'

Caused by:
ActiveRecord::RecordInvalid: Validation failed: Name has already been taken
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/validations.rb:80:in `raise_validation_error'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/validations.rb:53:in `save!'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:318:in `block in save!'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:375:in `block in with_transaction_returning_status'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/database_statements.rb:278:in `transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:212:in `transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:366:in `with_transaction_returning_status'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:318:in `save!'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/suppressor.rb:48:in `save!'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/persistence.rb:55:in `create!'
/usr/share/gems/gems/foreman_scc_manager-1.8.20/db/migrate/20210713092440_add_permissions.rb:8:in `up'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:831:in `exec_migration'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:812:in `block (2 levels) in migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:811:in `block in migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:471:in `with_connection'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:810:in `migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1002:in `migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1310:in `block in execute_migration_in_transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1361:in `block in ddl_transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/database_statements.rb:280:in `block in transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/transaction.rb:280:in `block in within_new_transaction'
/usr/share/gems/gems/activesupport-6.0.4.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
/usr/share/gems/gems/activesupport-6.0.4.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
/usr/share/gems/gems/activesupport-6.0.4.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
/usr/share/gems/gems/activesupport-6.0.4.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
/usr/share/gems/gems/activesupport-6.0.4.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/transaction.rb:278:in `within_new_transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/connection_adapters/abstract/database_statements.rb:280:in `transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/transactions.rb:212:in `transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1361:in `ddl_transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1309:in `execute_migration_in_transaction'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1281:in `block in migrate_without_lock'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1280:in `each'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1280:in `migrate_without_lock'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1229:in `block in migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1382:in `with_advisory_lock'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1229:in `migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1061:in `up'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/migration.rb:1036:in `migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/tasks/database_tasks.rb:238:in `migrate'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/railties/databases.rake:86:in `block (3 levels) in <top (required)>'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/railties/databases.rake:84:in `each'
/usr/share/gems/gems/activerecord-6.0.4.7/lib/active_record/railties/databases.rake:84:in `block (2 levels) in <top (required)>'
/usr/share/gems/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
Tasks: TOP => db:migrate
# rm -f /usr/share/gems/gems/foreman_scc_manager-1.8.20/db/migrate/20210713092440_add_permissions.rb
# foreman-rake db:migrate
== 20220425132605 AddSccKatelloRepositoryRelation: migrating ==================
-- create_table(:scc_katello_repositories)
   -> 0.0158s
== 20220425132605 AddSccKatelloRepositoryRelation: migrated (0.0158s) =========

== 20220429100843 RemoveRootRepositoryIdFromSccRepository: migrating ==========
-- remove_foreign_key(:scc_repositories, :katello_root_repositories, {:column=>:katello_root_repository_id, :on_delete=>:nullify})
   -> 0.0023s
-- remove_column(:scc_repositories, :katello_root_repository_id, :integer, {:index=>true, :null=>true})
   -> 0.0003s
== 20220429100843 RemoveRootRepositoryIdFromSccRepository: migrated (0.0026s) =

== 20220429102717 PopulateSccKatelloRepositories: migrating ===================
== 20220429102717 PopulateSccKatelloRepositories: migrated (0.0022s) ==========

== 20220513132652 PopulateUpstreamAuthenticationToken: migrating ==============
== 20220513132652 PopulateUpstreamAuthenticationToken: migrated (0.0182s) =====
# foreman-rake db:seed
# foreman-installer --scenario katello --skip-checks-i-know-better --disable-system-checks
nadjaheitmann commented 2 years ago

Not sure what exactly is happening here, but I suppose that foreman-installer runs adds all permissions from the engine file and the permissions are being added during db migration. The migration should not be used for new SCC manager installations I suppose. Here is the commit where the permissions have been adjusted:

https://github.com/ATIX-AG/foreman_scc_manager/commit/582bea6383158a6ad1d50a5a659ac5900fdac843

loopway commented 1 year ago

Same issue here. Any news on this? #106 has been closed without comment.

nadjaheitmann commented 1 year ago

@loopway @dgoetz I have opened a new PR #117 to fix the issue. Do you mind testing?

dgoetz commented 1 year ago

Back from vacation, so I found some time to test and it works fine. Thanks! Can you tag a new release? I can then handle packaging if you want.

nadjaheitmann commented 1 year ago

Can you tag a new release? I can then handle packaging if you want.

Both already done. The package is available in rpm/develop, I have also open PRs for rpm/3.4 and rpm/3.5.