search

ManageIQ / manageiq-providers-vmware

ManageIQ plugin for the VMware vSphere and vCloud providers.
Apache License 2.0
22 stars 70 forks source link

[Najdorf] unable to get template resource from provider #873

Closed szajac01 closed 1 year ago

szajac01 commented 1 year ago

Hi, We have an issue with VMware service catalogs getting cluster information from template. service_template = $evm.root['service_template'] service_resource = service_template.service_resources.first resource = service_resource.resource resource.eligible_clusters.each do |c| next unless c.tagged_with?('cluster_os', template.platform) values_hash[c.id] = c.name end

this line goes in rescue with log below

ERROR -- evm: [NoMethodError]: undefined methodroot' for nil:NilClass Method:[rescue in values_from_automate] [----] E, [2023-06-13T08:20:16.863237 #6537:131b4] ERROR -- evm: /var/www/miq/vmdb/app/models/dynamic_dialog_field_value_processor.rb:14:in values_from_automate' /var/www/miq/vmdb/app/models/dynamic_dialog_field_value_processor.rb:3:invalues_from_automate' /var/www/miq/vmdb/app/models/dialog_field.rb:185:in values_from_automate' /var/www/miq/vmdb/app/models/dialog_field_sorted_item.rb:117:inraw_values' /var/www/miq/vmdb/app/models/dialog_field_sorted_item.rb:68:in trigger_automate_value_updates' /var/www/miq/vmdb/app/models/dialog_field.rb:142:inupdate_and_serialize_values' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-api-1bd75ebbd839/app/controllers/api/base_controller/generic.rb:132:in block in refresh_dialog_fields_action' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-api-1bd75ebbd839/app/controllers/api/base_controller/generic.rb:129:ineach' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-api-1bd75ebbd839/app/controllers/api/base_controller/generic.rb:129:in refresh_dialog_fields_action' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-api-1bd75ebbd839/app/controllers/api/service_dialogs_controller.rb:96:inrefresh_dialog_fields_service_dialog' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-api-1bd75ebbd839/app/controllers/api/service_dialogs_controller.rb:18:in block in refresh_dialog_fields_resource' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-api-1bd75ebbd839/app/controllers/api/base_controller/action.rb:8:inapi_action' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-api-1bd75ebbd839/app/controllers/api/service_dialogs_controller.rb:14:in refresh_dialog_fields_resource' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-api-1bd75ebbd839/app/controllers/api/base_controller/manager.rb:101:inupdate_one_collection' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-api-1bd75ebbd839/app/controllers/api/base_controller/manager.rb:23:in update_collection' /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-api-1bd75ebbd839/app/controllers/api/base_controller.rb:97:inupdate' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_controller/metal/basic_implicit_render.rb:6:in send_action' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/abstract_controller/base.rb:195:inprocess_action' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_controller/metal/rendering.rb:30:in process_action' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/abstract_controller/callbacks.rb:42:inblock in process_action' /opt/manageiq/manageiq-gemset/gems/activesupport-6.0.5.1/lib/active_support/callbacks.rb:135:in run_callbacks' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/abstract_controller/callbacks.rb:41:inprocess_action' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_controller/metal/rescue.rb:22:in process_action' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_controller/metal/instrumentation.rb:33:inblock in process_action' /opt/manageiq/manageiq-gemset/gems/activesupport-6.0.5.1/lib/active_support/notifications.rb:180:in block in instrument' /opt/manageiq/manageiq-gemset/gems/activesupport-6.0.5.1/lib/active_support/notifications/instrumenter.rb:24:ininstrument' /opt/manageiq/manageiq-gemset/gems/activesupport-6.0.5.1/lib/active_support/notifications.rb:180:in instrument' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_controller/metal/instrumentation.rb:32:inprocess_action' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_controller/metal/params_wrapper.rb:245:in process_action' /opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/active_record/railties/controller_runtime.rb:27:inprocess_action' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/abstract_controller/base.rb:136:in process' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_controller/metal.rb:190:indispatch' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_controller/metal.rb:254:in dispatch' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_dispatch/routing/route_set.rb:50:indispatch' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_dispatch/routing/route_set.rb:33:in serve' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_dispatch/journey/router.rb:49:inblock in serve' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_dispatch/journey/router.rb:32:in each' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_dispatch/journey/router.rb:32:inserve' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_dispatch/routing/route_set.rb:834:in call' /var/www/miq/vmdb/lib/request_started_on_middleware.rb:12:incall' /opt/manageiq/manageiq-gemset/gems/rack-2.2.4/lib/rack/tempfile_reaper.rb:15:in call' /opt/manageiq/manageiq-gemset/gems/rack-2.2.4/lib/rack/etag.rb:27:incall' /opt/manageiq/manageiq-gemset/gems/rack-2.2.4/lib/rack/conditional_get.rb:40:in call' /opt/manageiq/manageiq-gemset/gems/rack-2.2.4/lib/rack/head.rb:12:incall' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_dispatch/http/content_security_policy.rb:18:in call' /opt/manageiq/manageiq-gemset/gems/rack-2.2.4/lib/rack/session/abstract/id.rb:266:incontext' /opt/manageiq/manageiq-gemset/gems/rack-2.2.4/lib/rack/session/abstract/id.rb:260:in call' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_dispatch/middleware/cookies.rb:654:incall' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_dispatch/middleware/callbacks.rb:27:in block in call' /opt/manageiq/manageiq-gemset/gems/activesupport-6.0.5.1/lib/active_support/callbacks.rb:101:inrun_callbacks' /opt/manageiq/manageiq-gemset/gems/actionpack-6.0.5.1/lib/action_dispatch/middleware/callbacks.rb:26:in call'

In the catalog item, when trying to uncheck placement, we have also an error (same if we try to create a new catalog item) image

Did someone got same behaviors and how to fix ?

szajac01 commented 1 year ago

we've got also errors like these

[----] E, [2023-06-13T10:16:30.170157 #32212:b7afd4b4] ERROR -- evm: Error when saving InventoryCollection:<GuestDevice>, strategy: local_db_find_missing_references with strategy: local_db_find_missing_references, saver_strategy: default, targeted: true. Message: can't quote InventoryRefresh::InventoryObjectLazy
[----] E, [2023-06-13T10:16:30.170388 #32212:b7afd4b4] ERROR -- evm: MIQ(ManageIQ::Providers::Vmware::InfraManager::Inventory::Saver#save_inventory) EMS: [xxxxxxxx], id: [2] Save Inventory failed
[----] E, [2023-06-13T10:16:30.170527 #32212:b7afd4b4] ERROR -- evm: [TypeError]: can't quote InventoryRefresh::InventoryObjectLazy  Method:[block (2 levels) in <class:LogProxy>]
[----] E, [2023-06-13T10:16:30.170716 #32212:b7afd4b4] ERROR -- evm: /opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/active_record/connection_adapters/abstract/quoting.rb:231:in `_quote'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/active_record/connection_adapters/postgresql/quoting.rb:144:in `_quote'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/active_record/connection_adapters/abstract/quoting.rb:18:in `quote'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/to_sql.rb:758:in `quote'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/to_sql.rb:710:in `quoted'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/to_sql.rb:85:in `visit_Arel_Nodes_Casted'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/visitor.rb:30:in `visit'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/to_sql.rb:610:in `visit_Arel_Nodes_Equality'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/visitor.rb:30:in `visit'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/to_sql.rb:789:in `block in inject_join'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/to_sql.rb:787:in `each'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/to_sql.rb:787:in `each_with_index'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/to_sql.rb:787:in `inject_join'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/to_sql.rb:577:in `visit_Arel_Nodes_And'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/visitor.rb:30:in `visit'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/visitors/visitor.rb:11:in `accept'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/active_record/connection_adapters/determine_if_preparable_visitor.rb:10:in `accept'
/opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/arel/nodes/node.rb:38:in `to_sql'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection/index/type/local_db.rb:121:in `block in build_multi_selection_condition'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection/index/type/local_db.rb:121:in `map'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection/index/type/local_db.rb:121:in `build_multi_selection_condition'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection/index/type/local_db.rb:251:in `db_relation'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection/index/type/local_db.rb:110:in `populate_index!'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection/index/type/local_db.rb:41:in `find'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection/index/proxy.rb:160:in `local_db_index_find'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection/index/proxy.rb:91:in `find'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection.rb:127:in `find'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_object_lazy.rb:150:in `load_object'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_object_lazy.rb:52:in `load'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_object.rb:67:in `block in attributes'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_object.rb:54:in `each'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_object.rb:54:in `attributes'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_collection/saver/base.rb:143:in `block in save!'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection/data_storage.rb:11:in `each'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection/data_storage.rb:11:in `each'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/inventory_collection.rb:104:in `each'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_collection/saver/base.rb:142:in `save!'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_collection/saver/base.rb:85:in `save_inventory_collection!'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_collection/base.rb:52:in `save_inventory'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_collection/base.rb:25:in `save_inventory_object_inventory'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_collection/topological_sort.rb:29:in `block (2 levels) in save_collections'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_collection/topological_sort.rb:28:in `each'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_collection/topological_sort.rb:28:in `block in save_collections'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_collection/topological_sort.rb:26:in `each'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_collection/topological_sort.rb:26:in `each_with_index'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_collection/topological_sort.rb:26:in `save_collections'
/opt/manageiq/manageiq-gemset/gems/inventory_refresh-0.2.3/lib/inventory_refresh/save_inventory.rb:20:in `save_inventory'
/var/www/miq/vmdb/app/models/manageiq/providers/inventory/persister.rb:23:in `persist!'
/opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-860d98b3f3d2/app/models/manageiq/providers/vmware/infra_manager/inventory/saver.rb:6:in `save_inventory'
/opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-860d98b3f3d2/app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb:413:in `block in save_inventory'
/opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-860d98b3f3d2/app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb:435:in `with_heartbeat'
/opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-860d98b3f3d2/app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb:412:in `save_inventory'
/opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-860d98b3f3d2/app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb:107:in `targeted_refresh'
/opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-860d98b3f3d2/app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb:62:in `vim_collector'
/opt/manageiq/manageiq-gemset/bundler/gems/manageiq-providers-vmware-860d98b3f3d2/app/models/manageiq/providers/vmware/infra_manager/inventory/collector.rb:48:in `block in vim_collector_thread'
szajac01 commented 1 year ago

error to fix is this one Some objects in vmdb are inconsistent with A nil class.

[----] E, [2023-06-13T17:01:36.940033 #211530:92f4] ERROR -- automation: MiqAeServiceModelBase.ar_method raised: <NoMethodError>: <undefined method `base_class' for NilClass:Class>
 [----] E, [2023-06-13T17:01:36.940168 #211530:92f4] ERROR -- automation: /var/www/miq/vmdb/app/models/miq_request_workflow.rb:983:in `block in convert_to_xml'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:981:in `each'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:981:in `convert_to_xml'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:985:in `block in convert_to_xml'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:981:in `each'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:981:in `convert_to_xml'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:985:in `block in convert_to_xml'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:981:in `each'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:981:in `convert_to_xml'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:985:in `block in convert_to_xml'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:981:in `each'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:981:in `convert_to_xml'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:985:in `block in convert_to_xml'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:981:in `each'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:981:in `convert_to_xml'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:967:in `get_ems_metadata_tree'
 /var/www/miq/vmdb/app/models/manageiq/providers/infra_manager/provision_workflow.rb:26:in `allowed_ci'
 /var/www/miq/vmdb/app/models/miq_request_workflow.rb:1116:in `allowed_clusters'
 /var/www/miq/vmdb/app/models/mixins/miq_provision_mixin.rb:81:in `block in eligible_resources'
 /var/www/miq/vmdb/app/models/mixins/miq_request_mixin.rb:158:in `workflow'
 /var/www/miq/vmdb/app/models/mixins/miq_provision_mixin.rb:72:in `eligible_resources'
 /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-automation_engine-58a081e7f737/lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_model_base.rb:348:in `public_send'
 /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-automation_engine-58a081e7f737/lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_model_base.rb:348:in `block in object_send'
 /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-automation_engine-58a081e7f737/lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_model_base.rb:367:in `ar_method'
 /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-automation_engine-58a081e7f737/lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_model_base.rb:383:in `ar_method'
 /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-automation_engine-58a081e7f737/lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_model_base.rb:346:in `object_send'
 /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-automation_engine-58a081e7f737/lib/miq_automation_engine/service_models/mixins/miq_ae_service_miq_provision_mixin.rb:46:in `eligible_resources'
 /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-automation_engine-58a081e7f737/lib/miq_automation_engine/service_models/mixins/miq_ae_service_miq_provision_mixin.rb:9:in `block (2 levels) in expose_eligible_resources'
 /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-automation_engine-58a081e7f737/lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_model_base.rb:367:in `ar_method'
 /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-automation_engine-58a081e7f737/lib/miq_automation_engine/engine/miq_ae_method_service/miq_ae_service_model_base.rb:383:in `ar_method'
 /opt/manageiq/manageiq-gemset/bundler/gems/manageiq-automation_engine-58a081e7f737/lib/miq_automation_engine/service_models/mixins/miq_ae_service_miq_provision_mixin.rb:8:in `block in expose_eligible_resources'
 (irb):7:in `rescue in irb_binding'
 (irb):6:in `irb_binding'
 /usr/share/gems/gems/irb-1.2.6/lib/irb/workspace.rb:114:in `eval'
 /usr/share/gems/gems/irb-1.2.6/lib/irb/workspace.rb:114:in `evaluate'
 /usr/share/gems/gems/irb-1.2.6/lib/irb/context.rb:459:in `evaluate'
 /usr/share/ruby/irb.rb:541:in `block (2 levels) in eval_input'
 /usr/share/ruby/irb.rb:704:in `signal_status'
 /usr/share/ruby/irb.rb:538:in `block in eval_input'
 /usr/share/ruby/irb/ruby-lex.rb:166:in `block (2 levels) in each_top_level_statement'
 /usr/share/ruby/irb/ruby-lex.rb:151:in `loop'
 /usr/share/ruby/irb/ruby-lex.rb:151:in `block in each_top_level_statement'
 /usr/share/ruby/irb/ruby-lex.rb:150:in `catch'
 /usr/share/ruby/irb/ruby-lex.rb:150:in `each_top_level_statement'
 /usr/share/ruby/irb.rb:537:in `eval_input'
 /usr/share/ruby/irb.rb:472:in `block in run'
 /usr/share/ruby/irb.rb:471:in `catch'
 /usr/share/ruby/irb.rb:471:in `run'
 /usr/share/ruby/irb.rb:400:in `start'
 /opt/manageiq/manageiq-gemset/gems/railties-6.0.5.1/lib/rails/commands/console/console_command.rb:70:in `start'
 /opt/manageiq/manageiq-gemset/gems/railties-6.0.5.1/lib/rails/commands/console/console_command.rb:19:in `start'
 /opt/manageiq/manageiq-gemset/gems/railties-6.0.5.1/lib/rails/commands/console/console_command.rb:102:in `perform'
 /opt/manageiq/manageiq-gemset/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
 /opt/manageiq/manageiq-gemset/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
 /opt/manageiq/manageiq-gemset/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
 /opt/manageiq/manageiq-gemset/gems/railties-6.0.5.1/lib/rails/command/base.rb:69:in `perform'
 /opt/manageiq/manageiq-gemset/gems/railties-6.0.5.1/lib/rails/command.rb:46:in `invoke'
 /opt/manageiq/manageiq-gemset/gems/railties-6.0.5.1/lib/rails/commands.rb:18:in `<main>'
 /opt/manageiq/manageiq-gemset/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
 /opt/manageiq/manageiq-gemset/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'

Should fix in /var/www/miq/vmdb/app/models/miq_request_workflow.rb

    980 def convert_to_xml(xml, result)
    981     result.each do |obj, children|
    982       next if obj.nil? # add this line to fix the issue
    983       @ems_xml_nodes[obj.class.base_class] ||= {}
    984       @ems_xml_nodes[obj.class.base_class][obj.id] = node = xml.add_element(obj.class.base_class.name, :object => ci_to_hash_struct(obj))
    985       convert_to_xml(node, children)
    986     end
    987 end
agrare commented 1 year ago

Hm I wonder if those refresh errors are possibly causing the other issue. I'd like to isolate the refresh issue if we can. Could you systemctl stop evmserverd if you can take some full downtime, otherwise you could move the vmware provider to a maintenance zone if you can't.

Then ssh to an appliance and run the following in a rails console:

# cd /var/www/miq/vmdb
# rails c
> ManageIQ::Providers::Vmware::InfraManager.find(2).queue_refresh
> MiqQueue.find_by(:class_name => "EmsRefresh", :method_name => "refresh", :queue_name => "ems_2")

And get check the logs to see if you see the same failure? If not can you re-check the Service Catalog Item UI?

szajac01 commented 1 year ago

Done , we have several workers so i was able to stop one.

irb(main):001:0> ManageIQ::Providers::Vmware::InfraManager.find(2).queue_refresh
PostgreSQLAdapter#log_after_checkout, connection_pool: size: 10, connections: 1, in use: 1, waiting_in_queue: 0
=> nil
irb(main):002:0> MiqQueue.find_by(:class_name => "EmsRefresh", :method_name => "refresh", :queue_name => "ems_2")
=> #<MiqQueue id: 76046825, target_id: nil, priority: 100, method_name: "refresh", state: "dequeue", created_on: "2023-06-19 16:42:07", updated_on: "2023-06-19 16:42:14", lock_version: 1, task_id: nil, deliver_on: nil, queue_name: "ems_2", class_name: "EmsRefresh", instance_id: nil, args: [], miq_callback: {}, msg_data: [FILTERED], zone: "myzone", role: "ems_inventory", server_guid: nil, msg_timeout: 7200, handler_id: 2153, handler_type: "MiqWorker", expires_on: nil, tracking_label: nil, user_id: nil, group_id: nil, tenant_id: nil, miq_task_id: nil>

**And get check the logs to see if you see the same failure? No errors after this...

[----] I, [2023-06-19T18:40:28.942591 #211896:93a8]  INFO -- evm: MIQ(ManageIQ::Providers::Vmware::InfraManager::MetricsCollectorWorker::Runner) ID [2068] PID [211896] GUID [4e9668ca-854c-4012-9f2e-69bb3e5d0b98] Request to exit received: Worker exiting.
[----] I, [2023-06-19T18:40:28.944789 #211896:93a8]  INFO -- evm: Deleting worker record for ManageIQ::Providers::Vmware::InfraManager::MetricsCollectorWorker, id 2068
[----] I, [2023-06-19T18:40:29.372285 #211818:92cc]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2023-06-19 16:40:29 UTC]...
[----] I, [2023-06-19T18:40:29.379678 #211818:92cc]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2023-06-19 16:40:29 UTC]...Complete
[----] I, [2023-06-19T18:40:29.380754 #211818:92cc]  INFO -- evm: MIQ(MiqServer::WorkerManagement::Systemd#quiesce_all_workers) Cleaning all active messages being processed by MiqServer
[----] I, [2023-06-19T18:41:55.410065 #421321:92f4]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for evm has been changed to [DEBUG]
[----] I, [2023-06-19T18:41:55.410313 #421321:92f4]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure has been changed to [WARN]
[----] I, [2023-06-19T18:41:55.410538 #421321:92f4]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim has been changed to [WARN]
[----] I, [2023-06-19T18:41:55.436725 #421321:92f4]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::ActiveRecordStore
[----] I, [2023-06-19T18:41:55.614576 #421321:92f4]  INFO -- evm: MIQ(Vmdb::Initializer.init) Initializing Application: Program Name: bin/rails, PID: 421321, ENV['EVMSERVER']:
[----] I, [2023-06-19T18:41:55.614704 #421321:92f4]  INFO -- evm: MIQ(Vmdb::Initializer.log_db_connectable) Successfully connected to the database.
[----] I, [2023-06-19T18:42:07.323904 #421321:92f4]  INFO -- evm: MIQ(MiqQueue.put) Message id: [76046825], Zone: [vmware_pxl], Role: [ems_inventory], Server: [], MiqTask id: [], Handler id: [], Ident: [ems_2], Target id: [], Instance id: [], Task id: [], Command: [EmsRefresh.refresh], Timeout: [7200], Priority: [100], State: [ready], Deliver On: [], Data: [1 bytes], Args: []

If not can you re-check the Service Catalog Item UI?**

As i said in a previous post we did fix the catalog issue changing the code in /var/www/miq/vmdb/app/models/miq_request_workflow.rb

    980 def convert_to_xml(xml, result)
    981     result.each do |obj, children|
    982       next if obj.nil? # add this line to fix the issue
    983       @ems_xml_nodes[obj.class.base_class] ||= {}
    984       @ems_xml_nodes[obj.class.base_class][obj.id] = node = xml.add_element(obj.class.base_class.name, :object => ci_to_hash_struct(obj))
    985       convert_to_xml(node, children)
    986     end
    987 end

But when the worker is up we still have errors like this in the evm.log

[----] E, [2023-06-19T13:41:14.154946 #211903:1330da988] ERROR -- evm: Error when saving InventoryCollection:<GuestDevice>, strategy: local_db_find_missing_references with strategy: local_db_find_missing_references, saver_strategy: default, targeted: true. Message: can't quote InventoryRefresh::InventoryObjectLazy
[----] E, [2023-06-19T13:41:14.155161 #211903:1330da988] ERROR -- evm: MIQ(ManageIQ::Providers::Vmware::InfraManager::Inventory::Saver#save_inventory) EMS: [my_provider], id: [2] Save Inventory failed
[----] E, [2023-06-19T13:41:14.155273 #211903:1330da988] ERROR -- evm: [TypeError]: can't quote InventoryRefresh::InventoryObjectLazy  Method:[block (2 levels) in <class:LogProxy>]
[----] E, [2023-06-19T13:41:14.155435 #211903:1330da988] ERROR -- evm: /opt/manageiq/manageiq-gemset/gems/activerecord-6.0.5.1/lib/active_record/connection_adapters/abstract/quoting.rb:231:in `_quote'
szajac01 commented 1 year ago

issue fixed by code change