ManageIQ / manageiq

ManageIQ Open-Source Management Platform
https://manageiq.org
Apache License 2.0
1.34k stars 900 forks source link

While the tenant is created, it suppose to be listing in cloud tenant page it may take up to four or five minutes. #22862

Open SanthoshRam-03 opened 5 months ago

SanthoshRam-03 commented 5 months ago

Hi @agrare ,

settingsourcecode

The automatic time interval of openstack is wanted to be reflected within less than 1 minute, without refreshing the power states in my case. and changed the ui in Settings-->ApplicationSettings-->Server: EVM [1] (current)-->Advanced (UI yaml) edited as :refresh_interval: 50.seconds and cannot remove the :allow_targeted_refresh: false,

settingui

When deleting manually :allow_targeted_refresh: false in UI, save the changes, refresh the page, and since it is not edited,

Can you please look into this?

agrare commented 5 months ago

Hey @SanthoshRam-03 by forcing full refreshes it is likely increasing the time it will take for new inventory to show up because we have to collect and save all of the existing inventory as well. Setting your refresh_interval to be less than the duration of a full refresh isn't doing anything other than ensuring that you are doing full refreshes constantly. Depending on the environment size a full refresh can take minutes.

agrare commented 5 months ago

I think deleting advanced settings is a different issue, I tried adding :test: test to the :api group so entirely unrelated to ems_refresh/openstack and I can't delete it via the UI cc @Fryguy @jeffibm

agrare commented 5 months ago

As a workaround you can clear all settings overrides for ems_refresh/openstack/allow_targeted_refresh by running the following in a rails console SettingsChange.where(:key => "/ems_refresh/openstack/allow_targeted_refresh").destroy_all

agrare commented 5 months ago

@SanthoshRam-03 you have to set the value to <<reset>>, see the top of the advanced settings page image

SanthoshRam-03 commented 4 months ago

Hi @agrare

I tried and cleared all settings overrides that you provided before using the Rails console, but they did not work as expected, and the creation of the tenant is taking more than 5 minutes, but in another default domain there are no issues; it's working fine.

The above issue in the non-default domain (non-working environment) has queue congestion (high volume of operations being performed or insufficient worker processes to handle the load) Database Performance (Slow database queries can impact the overall performance of ManageIQ),

So,this could be the reason for the delay in updating or syncing the tenants, in my opinion.

Please review the screenshot that I provided that contains some EVM logs for two different domains for your reference.

image (2)

Can you look into this?

SanthoshRam-03 commented 4 months ago

Hi @agrare before UI error of validation failed during refresh while refreshing the power states,

WorkingDefaultProvider_but_refresh_error

I made some changes in the source code to resolve the error:

File Path: ./app/models/manageiq/providers/inventory/persister/builder/cloud_manager.rb

def vm_and_miq_template_ancestry_save_block
  lambda do |_ems, inventory_collection|
    vms_inventory_collection = inventory_collection.dependency_attributes[:vms].try(:first)
    miq_templates_inventory_collection = inventory_collection.dependency_attributes[:miq_templates].try(:first)

    return if vms_inventory_collection.blank? || miq_templates_inventory_collection.blank?

    ActiveRecord::Base.transaction do
      # Process VMs
      vms_inventory_collection.data.each do |inventory_object|
        vm = inventory_object.object
        genealogy_parent = inventory_object.data[:genealogy_parent].load if inventory_object.data[:genealogy_parent]

        if genealogy_parent && vm && vm.id != genealogy_parent.id
          vm.update!(:genealogy_parent => genealogy_parent)
        else
          Rails.logger.warn("Skipping VM update because genealogy_parent is nil or VM is trying to be set as its own parent.")
        end
      end

      # Process Templates
      miq_templates_inventory_collection.data.each do |inventory_object|
        template = inventory_object.object
        genealogy_parent = inventory_object.data[:genealogy_parent].load if inventory_object.data[:genealogy_parent]

        if genealogy_parent && template && template.id != genealogy_parent.id
          template.update!(:genealogy_parent => genealogy_parent)
        else
          Rails.logger.warn("Skipping Template update because genealogy_parent is nil or Template is trying to be set as its own parent.")
        end
      end
    end
  end
end

In this code:

1.) I add a check to make sure both genealogy_parent and VM are not nil before attempting to update the VM. 2.) And check that both genealogy_parent and template are not nil before attempting to update the template. 3.) If either genealogy_parent, vm, or template is nil, it logs a warning and skips the update.

By changing this UI Last Refresh Status: Validation Failed error resolved, is this the correct way that I made it?

Probably, this could be any reason for the delay in updating the newly created cloud tenant.

agrare commented 4 months ago

That's extremely helpful thanks @SanthoshRam-03, do you see your added logging in the logs? Can you tell which if any vm/template are causing the issue?

SanthoshRam-03 commented 4 months ago

Hi @agrare I also don't know which vm/template is causing the error, and it is also not showing in the logs. Can you please provide me with details to find out the errored vm/template?

agrare commented 4 months ago

@SanthoshRam-03 can you try applying this?

      def vm_and_miq_template_ancestry_save_block
        lambda do |_ems, inventory_collection|
          vms_inventory_collection = inventory_collection.dependency_attributes[:vms].try(:first)
          miq_templates_inventory_collection = inventory_collection.dependency_attributes[:miq_templates].try(:first)

          return if vms_inventory_collection.blank? || miq_templates_inventory_collection.blank?

          # Fetch IDs of all vms and genealogy_parents, only if genealogy_parent is present
          vms_genealogy_parents = vms_inventory_collection.data.each_with_object({}) do |x, obj|
            unless x.data[:genealogy_parent].nil?
              genealogy_parent_id = x.data[:genealogy_parent].load.try(:id)
              obj[x.id] = genealogy_parent_id if genealogy_parent_id
            end
          end

          miq_template_genealogy_parents = miq_templates_inventory_collection.data.each_with_object({}) do |x, obj|
            unless x.data[:genealogy_parent].nil?
              genealogy_parent_id = x.data[:genealogy_parent].load.try(:id)
              obj[x.id] = genealogy_parent_id if genealogy_parent_id
            end
          end

          ActiveRecord::Base.transaction do
            # associate parent templates to child instances
            parent_miq_templates = miq_templates_inventory_collection.model_class
                                                                     .where(:id => vms_genealogy_parents.values).find_each.index_by(&:id)
            vms_inventory_collection.model_class
                                    .where(:id => vms_genealogy_parents.keys).find_each do |vm|
              if vm.id != vms_genealogy_parents[vm.id]
                $log.info("Setting genealogy_parent for VM id: [#{vm.id}] to id: [#{vms_genealogy_parents[vm.id]}]")
                vm.update!(:genealogy_parent => parent_miq_templates[vms_genealogy_parents[vm.id]])
              else
                $log.warn("Skipping genealogy_parent for VM id: [#{vm.id}] name: [#{vm.name}]")
              end
            end
          end

          ActiveRecord::Base.transaction do
            # associate parent instances to child templates
            parent_vms = vms_inventory_collection.model_class
                                                 .where(:id => miq_template_genealogy_parents.values).find_each.index_by(&:id)
            miq_templates_inventory_collection.model_class
                                              .where(:id => miq_template_genealogy_parents.keys).find_each do |miq_template|
              if miq_template.id != miq_template_genealogy_parents[miq_template.id]
                $log.info("Setting genealogy_parent for Template id: [#{miq_template.id}] to id: [#{miq_template_genealogy_parents[miq_template.id]}]")
                miq_template.update!(:genealogy_parent => parent_vms[miq_template_genealogy_parents[miq_template.id]])
              else
                $log.warn("Skipping genealogy_parent for Template id: [#{miq_template.id}] name: [#{miq_template.name}]")
              end
            end
          end
        end
      end
miq-bot commented 1 month ago

This issue has been automatically marked as stale because it has not been updated for at least 3 months.

If you can still reproduce this issue on the current release or on master, please reply with all of the information you have about it in order to keep the issue open.