search

ManageIQ / manageiq-providers-ibm_cloud

ManageIQ plugin for the IBM Cloud provider
Apache License 2.0
5 stars 36 forks source link

VPC: Fix child NetworkManager and StorageManager refresh #456

Closed jaywcarman closed 1 year ago

jaywcarman commented 1 year ago
jaywcarman commented 1 year ago

There was already spec coverage for this - I'm not sure why it was passing? https://github.com/ManageIQ/manageiq-providers-ibm_cloud/blob/master/spec/models/manageiq/providers/ibm_cloud/power_virtual_servers/cloud_manager/refresher_spec.rb#L37-L53

Before these patches, refreshing a NetworkManager fails with the following:

irb(main):001:0> EmsRefresh.refresh(ManageIQ::Providers::IbmCloud::VPC::NetworkManager.first)
PostgreSQLAdapter#log_after_checkout, connection_pool: size: 5, connections: 1, in use: 1, waiting_in_queue: 0
  ManageIQ::Providers::IbmCloud::VPC::NetworkManager Load (0.7ms)  SELECT "ext_management_systems".* FROM "ext_management_systems" WHERE "ext_management_systems"."type" = $1 ORDER BY "ext_management_systems"."id" ASC LIMIT $2  [["type", "ManageIQ::Providers::IbmCloud::VPC::NetworkManager"], ["LIMIT", 1]]
  ManageIQ::Providers::IbmCloud::VPC::NetworkManager Inst Including Associations (271.6ms - 1rows)
  ManageIQ::Providers::BaseManager Load (0.5ms)  SELECT "ext_management_systems".* FROM "ext_management_systems" WHERE "ext_management_systems"."type" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37, $38, $39, $40, $41, $42, $43, $44, $45, $46, $47, $48, $49, $50, $51, $52, $53, $54, $55, $56, $57, $58, $59, $60, $61, $62, $63, $64, $65, $66, $67, $68, $69, $70, $71, $72, $73, $74, $75, $76, $77) AND "ext_management_systems"."id" = $78 LIMIT $79  [["type", "ManageIQ::Providers::BaseManager"], ["type", "ManageIQ::Providers::NetworkManager"], ["type", "ManageIQ::Providers::IbmCloud::VPC::NetworkManager"], ["type", "ManageIQ::Providers::Amazon::NetworkManager"], ["type", "ManageIQ::Providers::Azure::NetworkManager"], ["type", "ManageIQ::Providers::AzureStack::NetworkManager"], ["type", "ManageIQ::Providers::Google::NetworkManager"], ["type", "ManageIQ::Providers::IbmCloud::PowerVirtualServers::NetworkManager"], ["type", "ManageIQ::Providers::Nsxt::NetworkManager"], ["type", "ManageIQ::Providers::Nuage::NetworkManager"], ["type", "ManageIQ::Providers::Openstack::NetworkManager"], ["type", "ManageIQ::Providers::IbmCic::NetworkManager"], ["type", "ManageIQ::Providers::IbmPowerVc::NetworkManager"], ["type", "ManageIQ::Providers::OracleCloud::NetworkManager"], ["type", "ManageIQ::Providers::Ovirt::NetworkManager"], ["type", "ManageIQ::Providers::Redhat::NetworkManager"], ["type", "ManageIQ::Providers::Vmware::NetworkManager"], ["type", "ManageIQ::Providers::AutomationManager"], ["type", "ManageIQ::Providers::EmbeddedAutomationManager"], ["type", "ManageIQ::Providers::EmbeddedAnsible::AutomationManager"], ["type", "ManageIQ::Providers::Workflows::AutomationManager"], ["type", "ManageIQ::Providers::ExternalAutomationManager"], ["type", "ManageIQ::Providers::AnsibleTower::AutomationManager"], ["type", "ManageIQ::Providers::Awx::AutomationManager"], ["type", "ManageIQ::Providers::CloudManager"], ["type", "ManageIQ::Providers::Amazon::CloudManager"], ["type", "ManageIQ::Providers::Azure::CloudManager"], ["type", "ManageIQ::Providers::AzureStack::CloudManager"], ["type", "ManageIQ::Providers::Google::CloudManager"], ["type", "ManageIQ::Providers::IbmCloud::PowerVirtualServers::CloudManager"], ["type", "ManageIQ::Providers::IbmCloud::VPC::CloudManager"], ["type", "ManageIQ::Providers::Openstack::CloudManager"], ["type", "ManageIQ::Providers::IbmCic::CloudManager"], ["type", "ManageIQ::Providers::IbmPowerVc::CloudManager"], ["type", "ManageIQ::Providers::OracleCloud::CloudManager"], ["type", "ManageIQ::Providers::Vmware::CloudManager"], ["type", "ManageIQ::Providers::ConfigurationManager"], ["type", "ManageIQ::Providers::Foreman::ConfigurationManager"], ["type", "ManageIQ::Providers::IbmTerraform::ConfigurationManager"], ["type", "ManageIQ::Providers::ContainerManager"], ["type", "ManageIQ::Providers::Kubernetes::ContainerManager"], ["type", "ManageIQ::Providers::Amazon::ContainerManager"], ["type", "ManageIQ::Providers::Azure::ContainerManager"], ["type", "ManageIQ::Providers::Google::ContainerManager"], ["type", "ManageIQ::Providers::IbmCloud::ContainerManager"], ["type", "ManageIQ::Providers::Openshift::ContainerManager"], ["type", "ManageIQ::Providers::OracleCloud::ContainerManager"], ["type", "ManageIQ::Providers::Vmware::ContainerManager"], ["type", "ManageIQ::Providers::InfraManager"], ["type", "ManageIQ::Providers::IbmPowerHmc::InfraManager"], ["type", "ManageIQ::Providers::Kubevirt::InfraManager"], ["type", "ManageIQ::Providers::Openstack::InfraManager"], ["type", "ManageIQ::Providers::Ovirt::InfraManager"], ["type", "ManageIQ::Providers::Redhat::InfraManager"], ["type", "ManageIQ::Providers::Vmware::InfraManager"], ["type", "ManageIQ::Providers::MonitoringManager"], ["type", "ManageIQ::Providers::Kubernetes::MonitoringManager"], ["type", "ManageIQ::Providers::Openshift::MonitoringManager"], ["type", "ManageIQ::Providers::PhysicalInfraManager"], ["type", "ManageIQ::Providers::CiscoIntersight::PhysicalInfraManager"], ["type", "ManageIQ::Providers::Lenovo::PhysicalInfraManager"], ["type", "ManageIQ::Providers::Redfish::PhysicalInfraManager"], ["type", "ManageIQ::Providers::ProvisioningManager"], ["type", "ManageIQ::Providers::Foreman::ProvisioningManager"], ["type", "ManageIQ::Providers::StorageManager"], ["type", "ManageIQ::Providers::Amazon::StorageManager::Ebs"], ["type", "ManageIQ::Providers::Amazon::StorageManager::S3"], ["type", "ManageIQ::Providers::Autosde::StorageManager"], ["type", "ManageIQ::Providers::IbmCloud::ObjectStorage::StorageManager"], ["type", "ManageIQ::Providers::IbmCloud::PowerVirtualServers::StorageManager"], ["type", "ManageIQ::Providers::IbmCloud::VPC::StorageManager"], ["type", "ManageIQ::Providers::Openstack::StorageManager::CinderManager"], ["type", "ManageIQ::Providers::IbmCic::StorageManager::CinderManager"], ["type", "ManageIQ::Providers::IbmPowerVc::StorageManager::CinderManager"], ["type", "ManageIQ::Providers::Openstack::StorageManager::SwiftManager"], ["type", "ManageIQ::Providers::IbmCic::StorageManager::SwiftManager"], ["type", "ManageIQ::Providers::IbmPowerVc::StorageManager::SwiftManager"], ["id", 9], ["LIMIT", 1]]
  ManageIQ::Providers::BaseManager Inst Including Associations (144.9ms - 1rows)
   (0.6ms)  SELECT relname FROM pg_class WHERE relkind = 'S' LIMIT 1
   (0.2ms)  SELECT last_value FROM accounts_id_seq
  ProviderTagMapping Load (0.3ms)  SELECT "provider_tag_mappings".* FROM "provider_tag_mappings" WHERE "provider_tag_mappings"."id" BETWEEN $1 AND $2  [["id", 0], ["id", 999999999999]]
  ProviderTagMapping Inst Including Associations (0.0ms - 0rows)
  TRANSACTION (0.2ms)  BEGIN
  Tenant Load (0.4ms)  SELECT "tenants".* FROM "tenants" WHERE "tenants"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
  Tenant Inst Including Associations (16.8ms - 1rows)
  Zone Load (0.4ms)  SELECT "zones".* FROM "zones" WHERE "zones"."id" = $1 LIMIT $2  [["id", 2], ["LIMIT", 1]]
  Zone Inst Including Associations (0.1ms - 1rows)
  MiqRegion Load (0.3ms)  SELECT "miq_regions".* FROM "miq_regions" WHERE "miq_regions"."region" = $1 LIMIT $2  [["region", 0], ["LIMIT", 1]]
  MiqRegion Inst Including Associations (0.1ms - 1rows)
  Zone Load (0.3ms)  SELECT "zones".* FROM "zones" WHERE "zones"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
  Zone Inst Including Associations (0.1ms - 1rows)
  ManageIQ::Providers::IbmCloud::VPC::NetworkManager Update (0.5ms)  UPDATE "ext_management_systems" SET "updated_on" = $1, "last_refresh_error" = $2, "last_refresh_date" = $3 WHERE "ext_management_systems"."id" = $4  [["updated_on", "2023-05-23 18:18:55.465179"], ["last_refresh_error", "expected cloud_database_flavors to be found subclassed as ManageIQ::Providers::IbmCloud::VPC::NetworkManager::CloudDatabaseFlavor, but instead found CloudDatabaseFlavor"], ["last_refresh_date", "2023-05-23 18:18:55.405591"], ["id", 10]]
  TRANSACTION (10.2ms)  COMMIT
/home/jwcarman/src/github.com/ManageIQ/manageiq/app/models/manageiq/providers/base_manager/refresher.rb:68:in `refresh': expected cloud_database_flavors to be found subclassed as ManageIQ::Providers::IbmCloud::VPC::NetworkManager::CloudDatabaseFlavor, but instead found CloudDatabaseFlavor (ManageIQ::Providers::BaseManager::Refresher::PartialRefreshError)
        from /home/jwcarman/src/github.com/ManageIQ/manageiq/app/models/manageiq/providers/base_manager/refresher.rb:11:in `refresh'
        from /home/jwcarman/src/github.com/ManageIQ/manageiq/app/models/ems_refresh.rb:74:in `block in refresh'
        from /home/jwcarman/src/github.com/ManageIQ/manageiq/app/models/ems_refresh.rb:73:in `each'
        from /home/jwcarman/src/github.com/ManageIQ/manageiq/app/models/ems_refresh.rb:73:in `refresh'
        from (irb):1:in `<main>'
        from /home/jwcarman/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-6.1.7.3/lib/rails/commands/console/console_command.rb:70:in `start'
        from /home/jwcarman/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-6.1.7.3/lib/rails/commands/console/console_command.rb:19:in `start'
        from /home/jwcarman/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-6.1.7.3/lib/rails/commands/console/console_command.rb:102:in `perform'
        from /home/jwcarman/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/thor-1.2.2/lib/thor/command.rb:27:in `run'
        from /home/jwcarman/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/thor-1.2.2/lib/thor/invocation.rb:127:in `invoke_command'
        from /home/jwcarman/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/thor-1.2.2/lib/thor.rb:392:in `dispatch'
        from /home/jwcarman/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-6.1.7.3/lib/rails/command/base.rb:69:in `perform'
        from /home/jwcarman/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-6.1.7.3/lib/rails/command.rb:48:in `invoke'
        from /home/jwcarman/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/railties-6.1.7.3/lib/rails/commands.rb:18:in `<main>'
        from /home/jwcarman/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
        from /home/jwcarman/.rbenv/versions/3.0.4/lib/ruby/gems/3.0.0/gems/bootsnap-1.16.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:32:in `require'
        ... 1 levels...
miq-bot commented 1 year ago

Checked commits https://github.com/jaywcarman/manageiq-providers-ibm_cloud/compare/d962b430e5d9045e32c1e326cea940f28da04739~...09593f46e37618d683499ebd423ec1a59b81c3b9 with ruby 2.6.10, rubocop 1.28.2, haml-lint 0.35.0, and yamllint 7 files checked, 0 offenses detected Everything looks fine. :cookie:

agrare commented 1 year ago

@jaywcarman I think the difference is in how you're running the refresh. You're passing the target directly to EmsRefresh.refresh() where we are delegating the #refresh and #refresh_queue methods. If you called ManageIQ::Providers::IbmCloud::VPC::NetworkManager.first.refresh I bet it would work.

jaywcarman commented 1 year ago

If you called ManageIQ::Providers::IbmCloud::VPC::NetworkManager.first.refresh I bet it would work.

Yes, that works, and I notice that it returns a {ManageIQ::Providers::IbmCloud::VPC::CloudManager::Refresher=>[#<ManageIQ::Providers::IbmCloud::VPC::CloudManager:0x00000000165c9098>], whereas the EmsRefresh.refresh will use the NetworkManager::Refresher and return a NetworkManager object.