theforeman / puppetdb_foreman

PuppetDB proxy in Foreman
http://theforeman.org
GNU General Public License v3.0
32 stars 23 forks source link

database columns potentially out of sync with foreman 3.4 #73

Closed madelaney closed 1 year ago

madelaney commented 2 years ago

When I attempted to setup a new Foreman(3.4.0) instance with puppetdb_foreman (v5.0.0-4). The database migration and seeding fail with a few missing columns:

2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): rake aborted!
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): ActiveModel::UnknownAttributeError: unknown attribute 'descrip
tion' for Setting::Puppetdb.
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/activemodel-6.1.6.1/
lib/active_model/attribute_assignment.rb:51:in `_assign_attribute'
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/activerecord-6.1.6.1
/lib/active_record/attribute_assignment.rb:21:in `block in _assign_attributes'
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/activerecord-6.1.6.1
/lib/active_record/attribute_assignment.rb:13:in `each'
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/activerecord-6.1.6.1
/lib/active_record/attribute_assignment.rb:13:in `_assign_attributes'
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/activemodel-6.1.6.1/
lib/active_model/attribute_assignment.rb:34:in `assign_attributes'
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/activerecord-6.1.6.1
/lib/active_record/core.rb:518:in `initialize'
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/activerecord-6.1.6.1
/lib/active_record/inheritance.rb:72:in `new'
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/activerecord-6.1.6.1
/lib/active_record/inheritance.rb:72:in `new'
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/activerecord-6.1.6.1
/lib/active_record/persistence.rb:54:in `create!'
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/puppetdb_foreman-5.0
.0/app/models/setting/puppetdb.rb:40:in `block (2 levels) in load_defaults'
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/puppetdb_foreman-5.0
.0/app/models/setting/puppetdb.rb:40:in `each'
2022-09-09 13:15:03 -0400 /Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]/returns (notice): /usr/share/foreman/vendor/ruby/2.7.0/gems/puppetdb_foreman-5.0
.0/app/models/setting/puppetdb.rb:40:in `block in load_defaults'

These fields appear to be removed in 20220124174632_drop_setting_fields.rb.

alexdga commented 1 year ago

Same problem here, can't upgrade to foreman 3.4 because of database migrations errors.

demonstrator commented 1 year ago

I have the same issue. Is there an ETA when this will be fixed as it's currently a blocker for foreman 3.4?

jeongheon81 commented 1 year ago

My solution, ansible code for workaround

# BUG:WORKAROUND: foreman-plugin-puppetdb settings DSL
- name: Install packages
  package:
    name:
      - "{{ 'rubygem-puppetdb_foreman' if ansible_distribution_major_version|int > 7 else 'tfm-rubygem-puppetdb_foreman' }}"
    state: present
  register: result
  until: result is success
  retries: 3
  delay: 3

- name: Fix foreman-plugin-puppetdb settings DSL
  shell:
    cmd: |
      if grep -q 'class Setting::Puppetdb < ::Setting' /usr/share/gems/gems/puppetdb_foreman-5.0.0/app/models/setting/puppetdb.rb ; then cat << '__EOF_PATCH__' | patch -N /usr/share/gems/gems/puppetdb_foreman-5.0.0/app/models/setting/puppetdb.rb
      --- /usr/share/gems/gems/puppetdb_foreman-5.0.0/app/models/setting/puppetdb.rb  2022-04-23 04:44:19.000000000 +0000
      +++ puppetdb.rb 2022-09-21 06:31:40.947521204 +0000
      @@ -1,48 +1,4 @@
      -class Setting::Puppetdb < ::Setting
      -  BLANK_ATTRS << 'puppetdb_address'
      -  BLANK_ATTRS << 'puppetdb_ssl_ca_file'
      -  BLANK_ATTRS << 'puppetdb_ssl_certificate'
      -  BLANK_ATTRS << 'puppetdb_ssl_private_key'
      -  BLANK_ATTRS << 'puppetdb_api_version'
      -
      -  def self.default_settings
      -    if SETTINGS[:puppetdb].present?
      -      default_enabled = SETTINGS[:puppetdb][:enabled]
      -      default_address = SETTINGS[:puppetdb][:address]
      -      default_ssl_ca_file = SETTINGS[:puppetdb][:ssl_ca_file]
      -      default_ssl_certificate = SETTINGS[:puppetdb][:ssl_certificate]
      -      default_ssl_private_key = SETTINGS[:puppetdb][:ssl_private_key]
      -      default_api_version = SETTINGS[:puppetdb][:api_version]
      -    end
      -
      -    default_enabled = false if default_enabled.nil?
      -    default_address ||= 'https://puppetdb:8081/pdb/cmd/v1'
      -    default_ssl_ca_file ||= (SETTINGS[:ssl_ca_file]).to_s
      -    default_ssl_certificate ||= (SETTINGS[:ssl_certificate]).to_s
      -    default_ssl_private_key ||= (SETTINGS[:ssl_priv_key]).to_s
      -    default_api_version ||= 4
      -
      -    [
      -      set('puppetdb_enabled', _("Integration with PuppetDB, enabled will deactivate a host in PuppetDB when it's deleted in Foreman"), default_enabled),
      -      set('puppetdb_address', _('Foreman will send PuppetDB requests to this address'), default_address),
      -      set('puppetdb_ssl_ca_file', _('Foreman will send PuppetDB requests with this CA file'), default_ssl_ca_file),
      -      set('puppetdb_ssl_certificate', _('Foreman will send PuppetDB requests with this certificate file'), default_ssl_certificate),
      -      set('puppetdb_ssl_private_key', _('Foreman will send PuppetDB requests with this key file'), default_ssl_private_key),
      -      set('puppetdb_api_version', _('Foreman will use this PuppetDB API version'), default_api_version, N_('PuppetDB API Version'), nil, collection: proc { ::Puppetdb::API_VERSIONS })
      -    ]
      -  end
      -
      -  def self.load_defaults
      -    # Check the table exists
      -    return unless super
      -
      -    transaction do
      -      default_settings.each { |s| create! s.update(:category => 'Setting::Puppetdb') }
      -    end
      -
      -    true
      -  end
      -
      +class Setting::Puppetdb
         def self.humanized_category
           N_('PuppetDB')
         end
      __EOF_PATCH__
      fi
      if grep -q 'puppetdb_foreman.load_default_settings' /usr/share/gems/gems/puppetdb_foreman-5.0.0/lib/puppetdb_foreman/engine.rb ; then cat << '__EOF_PATCH__' | patch -N /usr/share/gems/gems/puppetdb_foreman-5.0.0/lib/puppetdb_foreman/engine.rb
      --- /usr/share/gems/gems/puppetdb_foreman-5.0.0/lib/puppetdb_foreman/engine.rb  2022-04-23 04:44:19.000000000 +0000
      +++ engine.rb   2022-09-21 07:19:11.131072054 +0000
      @@ -2,14 +2,6 @@
         class Engine < ::Rails::Engine
           engine_name 'puppetdb_foreman'

      -    initializer 'puppetdb_foreman.load_default_settings', :before => :load_config_initializers do |_app|
      -      require_dependency File.expand_path('../../app/models/setting/puppetdb.rb', __dir__) if begin
      -                                                                                                     Setting.table_exists?
      -                                                                                                   rescue StandardError
      -                                                                                                     (false)
      -                                                                                                   end
      -    end
      -
           initializer 'puppetdb_foreman.load_app_instance_data' do |app|
             PuppetdbForeman::Engine.paths['db/migrate'].existent.each do |path|
               app.config.paths['db/migrate'] << path
      @@ -22,6 +14,47 @@

               apipie_documented_controllers ["#{PuppetdbForeman::Engine.root}/app/controllers/api/v2/*.rb"]

      +        settings do
      +          category :puppetdb, N_("PuppetDB") do
      +            setting "puppetdb_enabled",
      +              type: :boolean,
      +              default: false,
      +              full_name: N_("PuppetDB enabled"),
      +              description: N_("Integration with PuppetDB, enabled will deactivate a host in PuppetDB when it's deleted in Foreman")
      +
      +            setting "puppetdb_address",
      +              type: :string,
      +              default: "https://puppetdb:8081/pdb/cmd/v1",
      +              full_name: N_("PuppetDB address"),
      +              description: N_("Foreman will send PuppetDB requests to this address")
      +
      +            setting "puppetdb_ssl_ca_file",
      +              type: :string,
      +              default: (SETTINGS[:ssl_ca_file]).to_s,
      +              full_name: N_("PuppetDB address"),
      +              description: N_("Foreman will send PuppetDB requests with this CA file")
      +
      +            setting "puppetdb_ssl_certificate",
      +              type: :string,
      +              default: (SETTINGS[:ssl_certificate]).to_s,
      +              full_name: N_("PuppetDB address"),
      +              description: N_("Foreman will send PuppetDB requests with this certificate file")
      +
      +            setting "puppetdb_ssl_private_key",
      +              type: :string,
      +              default: (SETTINGS[:ssl_priv_key]).to_s,
      +              full_name: N_("PuppetDB address"),
      +              description: N_("Foreman will send PuppetDB requests with this key file")
      +
      +            setting "puppetdb_api_version",
      +              type: :string,
      +              collection: Proc.new { ::Puppetdb::API_VERSIONS },
      +              default: "4",
      +              full_name: N_("PuppetDB API Version"),
      +              description: N_("Foreman will use this PuppetDB API version")
      +          end
      +        end
      +
               security_block :puppetdb_foreman do
                 permission :view_puppetdb_nodes, :'puppetdb_foreman/nodes' => [:index, :show],
                                                  :'api/v2/puppetdb_nodes' => [:index, :unknown]
      __EOF_PATCH__
      fi
      if [ -e /usr/share/gems/gems/puppetdb_foreman-5.0.0/db/migrate ] ; then
        find /usr/share/gems/gems/puppetdb_foreman-5.0.0/db/migrate -name '*.rb' -exec mv {} {}.bak \;
      fi
    chdir: /root/
timogoebel commented 1 year ago

@kamils-iRonin provides a fix in https://github.com/theforeman/puppetdb_foreman/pull/74, do you guys mind testing it and giving feedback if this works? We can then merge it once ci passes and tests confirm the patch works and release a new version.