search

ManageIQ / manageiq

ManageIQ Open-Source Management Platform
https://manageiq.org
Apache License 2.0
1.35k stars 899 forks source link

Qemu - 503 Service Unavailable #21944

Closed kTipSSIoYv closed 1 year ago

kTipSSIoYv commented 2 years ago

There are no errors in the appliance console and all services are running. But I can't still connect to manageIQ UI. I always get 503 - Service unavailable. I'm using latest Morphy version of QEMU image.

Any idea what the issue is?

image

Screenshot of the Overview page of appliance console shows that there are no error.

image

I did see one error in the log.

[root@localhost ~]# tail -f /var/www/miq/vmdb/log/evm.log | grep -i error

Fryguy commented 2 years ago

@agrare I think possibly you verified qemu image? or perhaps it was @bdunne ?

agrare commented 2 years ago

@Fryguy I did but that was najdorf

agrare commented 2 years ago

Those errors are normal on startup

kTipSSIoYv commented 2 years ago

Any idea why I'm getting that error? I'm trying to find out if I can fix it. I'm trying to switch from docker to qemu.

agrare commented 2 years ago

@kTipSSIoYv can you check ps aux | grep MIQ ?

kTipSSIoYv commented 2 years ago

There's nothing. But the appliance shows that its running.

[root@localhost ~]# ps aux | grep MIQ
root        2030  0.0  0.0  12140  1032 pts/1    S+   15:49   0:00 grep --color=auto MIQ

image

agrare commented 2 years ago

That's very weird there are no workers or MIQ Server, can you upload as much of the evm.log as you can?

kTipSSIoYv commented 2 years ago

It is showing the process in ps aux. But still I'm unable to access it.

[root@localhost ~]# ps aux | grep MIQ root 2062 80.9 1.5 461864 242480 ? Rl 16:03 3:33 MIQ Server root 2129 0.0 0.0 12140 1172 pts/0 S+ 16:07 0:00 grep --color=auto MIQ

Here is the full log: 

[----] I, [2022-06-17T16:02:25.078340 #2008:2b122799794c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:02:25.086420 #2008:2b122799794c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:02:25.560090 #2008:2b122799794c]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:02:29.098555 #2008:2b122799794c]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: /opt/manageiq/manageiq-gemset/bin/rake, PID: 2008, ENV['EVMSERVER']: 
[----] I, [2022-06-17T16:02:46.541293 #2008:2b122799794c]  INFO -- evm: MIQ(EvmApplication.start) EVM Startup initiated
[----] I, [2022-06-17T16:04:59.664561 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:04:59.673439 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:05:00.168216 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:05:03.760528 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: /var/www/miq/vmdb/lib/workers/bin/evm_server.rb, PID: 2062, ENV['EVMSERVER']: true
[----] I, [2022-06-17T16:05:36.172715 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#validate_database) Database Adapter: [PostgreSQL], version: [PostgreSQL 10.19 on x86_64-redhat-linux-gnu, compiled by gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-7), 64-bit]
[----] I, [2022-06-17T16:05:38.786803 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding...
[----] I, [2022-06-17T16:05:38.807676 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqDatabase...
[----] I, [2022-06-17T16:05:39.149520 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqDatabase... Complete in 337.92752500005463ms
[----] I, [2022-06-17T16:05:39.151911 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqRegion...
[----] I, [2022-06-17T16:05:39.194891 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqRegion... Complete in 40.50585599998158ms
[----] I, [2022-06-17T16:05:39.196652 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqEnterprise...
[----] I, [2022-06-17T16:05:39.692221 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqEnterprise... Complete in 493.78346900005ms
[----] I, [2022-06-17T16:05:39.693886 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding Zone...
[----] I, [2022-06-17T16:05:39.763413 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding Zone... Complete in 67.84955800003445ms
[----] I, [2022-06-17T16:05:39.764999 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqServer...
[----] I, [2022-06-17T16:05:39.785902 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqServer... Complete in 18.996915000002446ms
[----] I, [2022-06-17T16:05:39.787737 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ServerRole...
[----] I, [2022-06-17T16:05:40.043873 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ServerRole... Complete in 254.3210960000124ms
[----] I, [2022-06-17T16:05:40.045436 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqWorkerType...
[----] I, [2022-06-17T16:06:13.197895 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqWorkerType... Complete in 33149.73531199996ms
[----] I, [2022-06-17T16:06:13.199838 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding Tenant...
[----] I, [2022-06-17T16:06:13.651036 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding Tenant... Complete in 449.4105190000255ms
[----] I, [2022-06-17T16:06:13.653309 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqProductFeature...
[----] I, [2022-06-17T16:06:17.627797 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqProductFeature... Complete in 3968.7977800000453ms
[----] I, [2022-06-17T16:06:17.629472 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqUserRole...
[----] I, [2022-06-17T16:06:21.707848 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqUserRole... Complete in 4076.4022969999587ms
[----] I, [2022-06-17T16:06:21.709617 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqGroup...
[----] I, [2022-06-17T16:06:22.718868 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqGroup... Complete in 1007.3452149999866ms
[----] I, [2022-06-17T16:06:22.720473 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding User...
[----] I, [2022-06-17T16:06:23.204367 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding User... Complete in 482.02635500001634ms
[----] I, [2022-06-17T16:06:23.207660 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqReport...
[----] I, [2022-06-17T16:06:24.342077 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqReport... Complete in 1131.8011500000011ms
[----] I, [2022-06-17T16:06:24.354808 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding... Complete in 45565.390273999925ms
[----] I, [2022-06-17T16:06:24.630120 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#check_migrations_up_to_date) database schema version 20210223085046 is up to date
[----] I, [2022-06-17T16:06:24.632366 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#impersonate_server) Impersonating server - id: 1000000000001, guid: e246d21e-61f4-4885-8072-f8d50e66b0f6
[----] I, [2022-06-17T16:06:37.446348 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#log_server_info) Server IP Address: 10.0.2.15
[----] I, [2022-06-17T16:06:37.448149 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#log_server_info) Server MAC Address: 52:54:00:12:34:56
[----] I, [2022-06-17T16:06:37.449700 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#log_server_info) Server GUID: e246d21e-61f4-4885-8072-f8d50e66b0f6
[----] I, [2022-06-17T16:06:37.451704 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#log_server_info) Server Zone: default
[----] I, [2022-06-17T16:06:37.583381 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#log_server_info) Server Role: automate,database_operations,database_owner,ems_inventory,ems_operations,event,remote_console,reporting,scheduler,smartstate,user_interface,web_services
[----] I, [2022-06-17T16:06:37.586240 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#log_server_info) Server Region number: 1, name: Region 1
[----] I, [2022-06-17T16:06:37.625874 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#log_server_info) Database Latency: 3.669229100000848 ms
[----] I, [2022-06-17T16:06:37.628329 #2062:2ac79ca3f94c]  INFO -- evm: *************************************************
[----] I, [2022-06-17T16:06:37.629579 #2062:2ac79ca3f94c]  INFO -- evm: * [VMDB] started on [2022-06-17 16:06:37 -0400] *
[----] I, [2022-06-17T16:06:37.630652 #2062:2ac79ca3f94c]  INFO -- evm: *************************************************
[----] I, [2022-06-17T16:06:37.636159 #2062:2ac79ca3f94c]  INFO -- evm: Release: Morphy
[----] I, [2022-06-17T16:06:37.638825 #2062:2ac79ca3f94c]  INFO -- evm: Version: morphy-1
[----] I, [2022-06-17T16:06:37.643943 #2062:2ac79ca3f94c]  INFO -- evm: Build:   20220210224352_1e24154
[----] I, [2022-06-17T16:06:37.645589 #2062:2ac79ca3f94c]  INFO -- evm: Codename: Morphy
[----] I, [2022-06-17T16:06:37.647251 #2062:2ac79ca3f94c]  INFO -- evm: RUBY Environment:  ruby 2.6.7p197 (2021-04-05 revision 67941) [x86_64-linux]
[----] I, [2022-06-17T16:06:37.649098 #2062:2ac79ca3f94c]  INFO -- evm: RAILS Environment: production version 6.0.4.4
[----] I, [2022-06-17T16:06:39.753285 #2062:2ac79ca3f94c]  INFO -- evm: VMDB settings:
[----] I, [2022-06-17T16:06:40.069118 #2062:2ac79ca3f94c]  INFO -- evm: 
---
:api:
  :token_ttl: 10.minutes
  :authentication_timeout: 30.seconds
  :max_results_per_page: 1000
:log:
  :level_api: info
  :level_automation: info
  :level_aws: info
  :level_ansible_tower: info
  :level_azure: warn
  :level_azure_stack: info
  :level_gce: info
  :level_ibm_cloud: info
  :level_lenovo: info
  :level_nsxt: info
  :level_nuage: info
  :level_fog: info
  :level_rhevm: info
  :level_scvmm: info
  :level_vcloud: info
  :level_vim: warn
  :collection:
    :include_automate_models_and_dialogs: true
    :archive:
      :pattern:
    :current:
      :pattern:
      - log/*.log
      - log/apache/*.log
      - log/*.txt
      - config/*
      - "/var/log/syslog*"
      - "/var/log/daemon.log*"
      - "/etc/default/ntp*"
      - "/var/log/messages*"
      - "/var/log/cron*"
      - "/var/log/tower/*.log"
      - BUILD
      - GUID
      - VERSION
    :ping_depot: true
    :ping_depot_timeout: 20
  :level: info
  :level_audit: info
  :level_rails: info
  :level_policy: info
  :level_remote_console: info
  :secret_filter: []
:ems:
  :ems_amazon:
    :event_handling:
      :event_groups:
        :addition:
          :critical:
          - AWS_EC2_Instance_CREATE
          - AWS_API_CALL_RunInstances
          - AWS_API_CALL_CreateTags
        :update:
          :critical:
          - AWS_EC2_Instance_UPDATE
        :deletion:
          :critical:
          - AWS_EC2_Instance_DELETE
          - AWS_API_CALL_TerminateInstances
        :power:
          :critical:
          - AWS_EC2_Instance_running
          - AWS_EC2_Instance_shutting-down
          - AWS_EC2_Instance_stopped
          - EC2_Instance_State_change_Notification_running
          - EC2_Instance_State_change_Notification_shutting_down
          - EC2_Instance_State_change_Notification_stopped
          - EC2_Instance_State_change_Notification_stopping
          - EC2_Instance_State_change_Notification_terminated
          - EC2_Instance_State_change_Notification_pending
          - AWS_API_CALL_StopInstances
          - AWS_API_CALL_StartInstances
    :blacklisted_event_names:
    - ConfigurationSnapshotDeliveryCompleted
    - ConfigurationSnapshotDeliveryStarted
    - ConfigurationSnapshotDeliveryFailed
    :disabled_regions: []
    :additional_regions: {}
    :additional_instance_types: {}
    :disabled_instance_types: []
    :agent_coordinator:
      :agent_ami_login_user: centos
      :agent_ami_name: CentOS Atomic Host 7 x86_64 HVM EBS 1706_01
      :agent_idle_period: 900
      :agent_label: smartstate
      :docker_image: manageiq/amazon-smartstate:latest-morphy
      :docker_login_required: false
      :docker_registry:
      :heartbeat_interval: 120
      :log_level: INFO
      :response_thread_sleep_seconds: 10
  :ems_ansible_tower:
    :blacklisted_event_names: []
    :event_handling:
      :event_groups:
  :ems_autosde:
    :blacklisted_event_names: []
    :event_handling:
      :event_groups:
        :power:
          :critical:
          - AUTOSDE_instance_power_on
          - AUTOSDE_instance_power_off
  :ems_azure:
    :api_versions:
      :availability_set: '2017-12-01'
      :ip_address: '2017-11-01'
      :load_balancer: '2017-11-01'
      :managed_image: '2017-12-01'
      :network_interface: '2017-11-01'
      :network_security_group: '2017-11-01'
      :resource: '2017-08-01'
      :resource_group: '2017-08-01'
      :resource_provider: '2016-09-01'
      :route_table: '2018-03-01'
      :storage_account: '2017-10-01'
      :storage_disk: '2017-03-30'
      :template_deployment: '2017-08-01'
      :virtual_machine: '2017-12-01'
      :virtual_network: '2017-11-01'
    :blacklisted_event_names:
    - storageAccounts_listKeys_BeginRequest
    - storageAccounts_listKeys_EndRequest
    - deployments_exportTemplate_BeginRequest
    - deployments_exportTemplate_EndRequest
    :event_handling:
      :event_groups:
        :addition:
          :critical:
          - virtualMachines_write_EndRequest
        :deletion:
          :critical:
          - virtualMachines_delete_EndRequest
        :power:
          :critical:
          - virtualMachines_deallocate_EndRequest
          - virtualMachines_start_EndRequest
          - virtualMachines_restart_EndRequest
    :disabled_regions: []
    :additional_regions: {}
  :ems_azure_stack:
    :api_connection_timeout: 5.seconds
    :blacklisted_event_names: []
    :event_handling:
      :event_groups:
        :update:
          :critical:
          - !ruby/regexp /^Administrative_Microsoft.(Compute|Resources)_.+_write_(Succeeded|Failed)$/
          :detail:
          - !ruby/regexp /^Administrative_Microsoft.(Compute|Resources)_.+_write_.+$/
        :deletion:
          :critical:
          - !ruby/regexp /^Administrative_Microsoft.(Compute|Resources)_.+_delete_(Succeeded|Failed)$/
          :detail:
          - !ruby/regexp /^Administrative_Microsoft.(Compute|Resources)_.+_delete_.+$/
        :power:
          :critical:
          - !ruby/regexp /^Administrative_Microsoft.(Compute|Resources)_.+_(restart|start|deallocate)_(Succeeded|Failed)$/
          :detail:
          - !ruby/regexp /^Administrative_Microsoft.(Compute|Resources)_.+_(restart|start|deallocate)_.+$/
        :status:
          :critical:
          - !ruby/regexp /^(ServiceHealth|ResourceHealth|Alert|AutoScale|Security|Recommendation)_.+$/
        :storage:
          :critical:
          - !ruby/regexp /^Administrative_Microsoft.Storage_.+$/
        :network:
          :critical:
          - !ruby/regexp /^Administrative_Microsoft.Network_.+$/
  :ems_foreman:
    :blacklisted_event_names: []
    :event_handling:
      :event_groups:
  :ems_google:
    :blacklisted_event_names: []
    :event_handling:
      :event_groups:
  :ems_ibm_cloud_power_virtual_servers:
    :api_endpoint_overrides:
      :dal12: us-south.power-iaas.cloud.ibm.com
    :event_handling:
      :event_groups:
        :power:
          :critical:
          - IBM_CLOUD_POWER_VIRTUAL_SERVERS_instance_power_on
          - IBM_CLOUD_POWER_VIRTUAL_SERVERS_instance_power_off
  :ems_ibm_cloud_vpc:
    :event_handling:
      :event_groups:
        :addition:
          :critical:
          - is.instance.instance.create
          - is.instance.action.create
          - is.instance.console-access-token.create
          - is.instance.volume-attachment.create
          - is.instance.network-interface.create
        :update:
          :critical:
          - is.instance.instance.update
          - is.instance.instance.read
          - is.instance.console.read
          - is.instance.network-interface_floating-ip.attach
          - is.instance.network-interface_floating-ip.detach
          - is.instance.volume-attachment.update
          - is.instance.volume-attachment.read
          - is.instance.network-interface.update
          - is.instance.network-interface.read
          - is.instance.network-interface_floating-ip.read
          - is.instance.disk.read
          - is.instance.disk.update
        :deletion:
          :critical:
          - is.instance.instance.delete
          - is.instance.action.delete
          - is.instance.volume-attachment.delete
          - is.instance.network-interface.delete
          - is.instance.disk.wipe
          - is.instance.gpu.wipe
        :power:
          :critical:
          - is.instance.instance.start
          - is.instance.instance.stop
          - is.instance.instance.volume-stop
  :ems_ibm_power_vc:
    :excon:
      :omit_default_port: true
      :read_timeout: 60
    :blacklisted_event_names: []
    :event_handling:
      :event_backread_seconds: 5
      :event_skip_history: false
      :event_groups:
        :addition:
          :critical:
          - aggregate.addhost.end
          - aggregate.create.end
          - aggregate.removehost.end
          - aggregate.updateprop.end
          - aggregate.updatemetadata.end
          - identity.project.created
          - identity.project.deleted
          - identity.project.updated
          - image.update
          - image.create
          - image.upload
          - orchestration.stack.create.end
          - orchestration.stack.create.error
          - servergroup.create
        :configurat...
[----] I, [2022-06-17T16:06:40.070942 #2062:2ac79ca3f94c]  INFO -- evm: VMDB settings END
[----] I, [2022-06-17T16:06:40.071971 #2062:2ac79ca3f94c]  INFO -- evm: ---
[----] I, [2022-06-17T16:06:40.072772 #2062:2ac79ca3f94c]  INFO -- evm: DATABASE settings:
[----] I, [2022-06-17T16:06:40.077084 #2062:2ac79ca3f94c]  INFO -- evm: 
---
:adapter: postgresql
:encoding: utf8
:username: root
:pool: 5
:wait_timeout: 5
:min_messages: warning
:database: vmdb_production
:host: localhost
:password: [FILTERED]
[----] I, [2022-06-17T16:06:40.078260 #2062:2ac79ca3f94c]  INFO -- evm: DATABASE settings END
[----] I, [2022-06-17T16:06:40.079236 #2062:2ac79ca3f94c]  INFO -- evm: ---
[----] I, [2022-06-17T16:06:42.108484 #2062:2ac79ca3f94c]  INFO -- evm: Diagnostics: [Uptime, top processes, and memory usage]
top - 16:06:42 up 12 min,  3 users,  load average: 1.16, 1.28, 0.90
Tasks: 188 total,   1 running, 187 sleeping,   0 stopped,   0 zombie
%Cpu(s):  6.9 us, 13.8 sy,  0.0 ni, 77.6 id,  0.0 wa,  0.0 hi,  1.7 si,  0.0 st
MiB Mem :  14819.7 total,  13717.0 free,    658.4 used,    444.3 buff/cache
MiB Swap:   6140.0 total,   6140.0 free,      0.0 used.  13821.1 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
   2105 root      20   0   65432   4352   3772 R  45.5   0.0   0:00.25 top
   1271 root      20   0  496180  31296  15364 S   4.5   0.2   0:12.98 tuned
      1 root      20   0  241036  14040   9088 S   0.0   0.1   0:13.30 systemd
      2 root      20   0       0      0      0 S   0.0   0.0   0:00.05 kthreadd
      3 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 rcu_gp
      4 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 rcu_par+
      6 root       0 -20       0      0      0 I   0.0   0.0   0:00.85 kworker+
      7 root      20   0       0      0      0 I   0.0   0.0   0:00.42 kworker+
      8 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 mm_perc+
      9 root      20   0       0      0      0 S   0.0   0.0   0:00.00 rcu_tas+
     10 root      20   0       0      0      0 S   0.0   0.0   0:00.00 rcu_tas+
     11 root      20   0       0      0      0 S   0.0   0.0   0:00.82 ksoftir+
     12 root      20   0       0      0      0 I   0.0   0.0   0:05.25 rcu_sch+
     13 root      rt   0       0      0      0 S   0.0   0.0   0:00.01 migrati+
     14 root      rt   0       0      0      0 S   0.0   0.0   0:00.01 watchdo+
     15 root      20   0       0      0      0 S   0.0   0.0   0:00.00 cpuhp/0
     16 root      20   0       0      0      0 S   0.0   0.0   0:00.00 cpuhp/1
     17 root      rt   0       0      0      0 S   0.0   0.0   0:00.01 watchdo+
     18 root      rt   0       0      0      0 S   0.0   0.0   0:00.01 migrati+
     19 root      20   0       0      0      0 S   0.0   0.0   0:01.26 ksoftir+
     21 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kworker+
     24 root      20   0       0      0      0 S   0.0   0.0   0:00.02 kdevtmp+
     25 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 netns
     26 root      20   0       0      0      0 S   0.0   0.0   0:00.02 kauditd
     27 root      20   0       0      0      0 I   0.0   0.0   0:00.50 kworker+
     29 root      20   0       0      0      0 S   0.0   0.0   0:00.00 khungta+
     30 root      20   0       0      0      0 S   0.0   0.0   0:00.00 oom_rea+
     31 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 writeba+
     32 root      20   0       0      0      0 S   0.0   0.0   0:00.00 kcompac+
     33 root      25   5       0      0      0 S   0.0   0.0   0:00.00 ksmd
     34 root      39  19       0      0      0 S   0.0   0.0   0:00.09 khugepa+
     35 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 crypto
     36 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kintegr+
     37 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kblockd
     38 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 blkcg_p+
     39 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 tpm_dev+
     40 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 md
     41 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 edac-po+
     42 root      rt   0       0      0      0 S   0.0   0.0   0:00.00 watchdo+
     43 root       0 -20       0      0      0 I   0.0   0.0   0:00.34 kworker+
     46 root      20   0       0      0      0 S   0.0   0.0   0:00.00 kswapd0
    148 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kthrotld
    149 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 acpi_th+
    150 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kmpath_+
    151 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kaluad
    153 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 ipv6_ad+
    154 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kstrp
    171 root       0 -20       0      0      0 I   0.0   0.0   0:00.23 kworker+
    223 root      20   0       0      0      0 I   0.0   0.0   0:00.51 kworker+
    224 root      20   0       0      0      0 I   0.0   0.0   0:00.47 kworker+
    373 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 ata_sff
    376 root      20   0       0      0      0 S   0.0   0.0   0:00.01 scsi_eh+
    377 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 scsi_tm+
    378 root      20   0       0      0      0 S   0.0   0.0   0:00.03 scsi_eh+
    379 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 scsi_tm+
    381 root      20   0       0      0      0 I   0.0   0.0   0:00.29 kworker+
    472 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kdmflus+
    481 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kdmflus+
    509 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfsalloc
    510 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs_mru+
    511 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-buf+
    512 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-con+
    513 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-cil+
    514 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-rec+
    515 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-eof+
    516 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-log+
    517 root      20   0       0      0      0 S   0.0   0.0   0:00.18 xfsaild+
    624 root      20   0   97672  12236  11056 S   0.0   0.1   0:02.31 systemd+
    655 root      20   0  100560  10152   7752 S   0.0   0.1   0:01.74 systemd+
    656 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-buf+
    657 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-con+
    658 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-cil+
    659 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-rec+
    660 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-eof+
    661 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-log+
    662 root      20   0       0      0      0 S   0.0   0.0   0:00.00 xfsaild+
    706 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kdmflus+
    707 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kdmflus+
    708 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kdmflus+
    709 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kdmflus+
    711 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kdmflus+
    713 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kdmflus+
    741 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-buf+
    742 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-con+
    743 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-cil+
    744 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-rec+
    745 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-eof+
    746 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-log+
    747 root      20   0       0      0      0 S   0.0   0.0   0:00.00 xfsaild+
    748 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-buf+
    749 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-con+
    750 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-cil+
    751 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-rec+
    752 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-eof+
    753 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 xfs-log+
    754 root      20   0       0      0      0 S   0.0   0.0   0:00.08 xfsaild+
    756 root       0 -20       0      0   ...
[----] I, [2022-06-17T16:06:42.412827 #2062:2ac79ca3f94c]  INFO -- evm: Diagnostics: [Process tree]
systemd,1 --switched-root --system --deserialize 18
  |-NetworkManager,998 --no-daemon
  |   |-{NetworkManager},1003
  |   `-{NetworkManager},1005
  |-agetty,1502 -o -p -- \\u --keep-baud 115200,38400,9600 ttyS0 vt220
  |-atd,1495 -f
  |-auditd,830
  |   |-sedispatch,832
  |   |-{auditd},831
  |   `-{auditd},833
  |-chronyd,895
  |-crond,1492 -n
  |-dbus-daemon,877 --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
  |   `-{dbus-daemon},887
  |-evm_watchdog,881 /bin/evm_watchdog
  |   `-ruby,886 /bin/evm_watchdog.rb
  |-firewalld,924 -s /usr/sbin/firewalld --nofork --nopid
  |   `-{firewalld},1053
  |-gssproxy,1274 -D
  |   |-{gssproxy},1276
  |   |-{gssproxy},1277
  |   |-{gssproxy},1278
  |   |-{gssproxy},1279
  |   `-{gssproxy},1280
  |-irqbalance,883 --foreground
  |   `-{irqbalance},888
  |-login,1501
  |   `-bash,1865
  |       `-appliance_conso,1893 /opt/manageiq/manageiq-gemset/bin/appliance_console
  |-mcelog,878 --ignorenodev --daemon --foreground
  |-memcached,1272 -p 11211 -u memcached -m 64 -c 1024 -l 127.0.0.1 -I 1M
  |   |-{memcached},1285
  |   |-{memcached},1286
  |   |-{memcached},1287
  |   |-{memcached},1288
  |   |-{memcached},1289
  |   |-{memcached},1290
  |   |-{memcached},1291
  |   |-{memcached},1292
  |   `-{memcached},1293
  |-polkitd,876 --no-debug
  |   |-{polkitd},902
  |   |-{polkitd},905
  |   |-{polkitd},909
  |   |-{polkitd},910
  |   `-{polkitd},922
  |-postmaster,1282 -D /var/lib/pgsql/data
  |   |-postmaster,1294
  |   |-postmaster,1349
  |   |-postmaster,1350
  |   |-postmaster,1351
  |   |-postmaster,1352
  |   |-postmaster,1353
  |   |-postmaster,1354
  |   `-postmaster,2076
  |-rpcbind,828 -w -f
  |-rsyslogd,1480 -n
  |   |-{rsyslogd},1496
  |   `-{rsyslogd},1500
  |-ruby,2062
  |   |-pstree,2106 -a -p
  |   |-{ruby},2071
  |   |-{ruby},2093
  |   |-{ruby},2094
  |   `-{ruby},2095
  |-sshd,1484 -D-oCiphers=aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128
  |   |-sshd,1895
  |   |   `-sshd,1906
  |   |       `-bash,1907
  |   |           `-tail,2066 -f /var/www/miq/vmdb/log/evm.log
  |   `-sshd,1933
  |       `-sshd,1937
  |           `-bash,1938
  |               `-tail,2001 -f /var/www/miq/vmdb/log/evm.log
  |-sssd,882 -i --logger=files
  |   |-sssd_be,921 --domain implicit_files --uid 0 --gid 0 --logger=files
  |   `-sssd_nss,925 --uid 0 --gid 0 --logger=files
  |-sssd_kcm,1973 --uid 0 --gid 0 --logger=files
  |-systemd,1856 --user
  |   `-(sd-pam),1858
  |-systemd-journal,624
  |-systemd-logind,926
  |-systemd-udevd,655
  |-top,919 -b -c -d 60
  |-tuned,1271 -Es /usr/sbin/tuned -l -P
  |   |-{tuned},1437
  |   |-{tuned},1449
  |   |-{tuned},1450
  |   `-{tuned},1451
  `-vmstat,918 -a -n 60

[----] I, [2022-06-17T16:06:42.824965 #2062:2ac79ca3f94c]  INFO -- evm: Diagnostics: [File system disk usage]
Filesystem                             Type        Size  Used Avail Use% Mounted on
sysfs                                  sysfs          0     0     0    - /sys
proc                                   proc           0     0     0    - /proc
devtmpfs                               devtmpfs    7.3G     0  7.3G   0% /dev
securityfs                             securityfs     0     0     0    - /sys/kernel/security
tmpfs                                  tmpfs       7.3G  124K  7.3G   1% /dev/shm
devpts                                 devpts         0     0     0    - /dev/pts
tmpfs                                  tmpfs       7.3G  540K  7.3G   1% /run
tmpfs                                  tmpfs       7.3G     0  7.3G   0% /sys/fs/cgroup
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/systemd
pstore                                 pstore         0     0     0    - /sys/fs/pstore
bpf                                    bpf            0     0     0    - /sys/fs/bpf
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/freezer
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/rdma
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/cpuset
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/perf_event
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/memory
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/devices
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/cpu,cpuacct
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/blkio
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/net_cls,net_prio
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/pids
cgroup                                 cgroup         0     0     0    - /sys/fs/cgroup/hugetlb
none                                   tracefs        0     0     0    - /sys/kernel/tracing
configfs                               configfs       0     0     0    - /sys/kernel/config
/dev/mapper/vg_system-lv_os            xfs          11G  3.0G  7.6G  28% /
selinuxfs                              selinuxfs      0     0     0    - /sys/fs/selinux
systemd-1                              autofs         0     0     0    - /proc/sys/fs/binfmt_misc
mqueue                                 mqueue         0     0     0    - /dev/mqueue
hugetlbfs                              hugetlbfs      0     0     0    - /dev/hugepages
debugfs                                debugfs        0     0     0    - /sys/kernel/debug
/dev/sda1                              xfs        1014M  215M  800M  22% /boot
/dev/mapper/vg_system-lv_home          xfs        1014M   40M  975M   4% /home
/dev/mapper/vg_system-lv_tmp           xfs        1014M   40M  975M   4% /tmp
/dev/mapper/vg_system-lv_var           xfs          12G  836M   12G   7% /var
/dev/sda3                              xfs          10G  104M  9.9G   2% /var/www/miq_tmp
/dev/mapper/vg_data-lv_pg              xfs          13G  230M   13G   2% /var/lib/pgsql
/dev/mapper/vg_system-lv_var_log       xfs          11G  179M   11G   2% /var/log
/dev/mapper/vg_system-lv_var_log_audit xfs         507M   30M  477M   6% /var/log/audit
sunrpc                                 rpc_pipefs     0     0     0    - /var/lib/nfs/rpc_pipefs
tmpfs                                  tmpfs       1.5G     0  1.5G   0% /run/user/0

[----] I, [2022-06-17T16:06:42.953222 #2062:2ac79ca3f94c]  INFO -- evm: Diagnostics: [Mounted file systems]
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=7566976k,nr_inodes=1891744,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,freezer)
cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,rdma)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpuset)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,perf_event)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,devices)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,cpu,cpuacct)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,blkio)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,net_cls,net_prio)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,pids)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,seclabel,hugetlb)
none on /sys/kernel/tracing type tracefs (rw,relatime,seclabel)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/mapper/vg_system-lv_os on / type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14890)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel,pagesize=2M)
debugfs on /sys/kernel/debug type debugfs (rw,relatime,seclabel)
/dev/sda1 on /boot type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/mapper/vg_system-lv_home on /home type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/mapper/vg_system-lv_tmp on /tmp type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/mapper/vg_system-lv_var on /var type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/sda3 on /var/www/miq_tmp type xfs (rw,noatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/mapper/vg_data-lv_pg on /var/lib/pgsql type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/mapper/vg_system-lv_var_log on /var/log type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/mapper/vg_system-lv_var_log_audit on /var/log/audit type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=1517540k,mode=700)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)

[----] I, [2022-06-17T16:06:43.058494 #2062:2ac79ca3f94c]  INFO -- evm: Diagnostics: [Currently active interfaces]
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.2.15  netmask 255.255.255.0  broadcast 10.0.2.255
        inet6 fec0::5054:ff:fe12:3456  prefixlen 64  scopeid 0x40<site>
        inet6 fe80::5054:ff:fe12:3456  prefixlen 64  scopeid 0x20<link>
        ether 52:54:00:12:34:56  txqueuelen 1000  (Ethernet)
        RX packets 580  bytes 50072 (48.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 459  bytes 131718 (128.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 2078  bytes 2257036 (2.1 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2078  bytes 2257036 (2.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[----] I, [2022-06-17T16:06:43.248153 #2062:2ac79ca3f94c]  INFO -- evm: Diagnostics: [IP Routing table]
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         _gateway        0.0.0.0         UG    100    0        0 eth0
10.0.2.0        0.0.0.0         255.255.255.0   U     100    0        0 eth0

[----] I, [2022-06-17T16:06:43.351210 #2062:2ac79ca3f94c]  INFO -- evm: Diagnostics: [Network interface table]
Kernel Interface table
Iface             MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0             1500      586      0      0 0           465      0      0      0 BMRU
lo              65536     2078      0      0 0          2078      0      0      0 LRU

[----] I, [2022-06-17T16:06:43.447815 #2062:2ac79ca3f94c]  INFO -- evm: Diagnostics: [Network statistics]
Ip:
    Forwarding: 2
    607 total packets received
    0 forwarded
    0 incoming packets discarded
    606 incoming packets delivered
    448 requests sent out
    2 dropped because of missing route
Icmp:
    0 ICMP messages received
    0 input ICMP message failed
    ICMP input histogram:
    0 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
Tcp:
    14 active connection openings
    13 passive connection openings
    0 failed connection attempts
    11 connection resets received
    6 connections established
    2319 segments received
    2187 segments sent out
    4 segments retransmitted
    0 bad segments received
    7 resets sent
Udp:
    57 packets received
    0 packets to unknown port received
    0 packet receive errors
    57 packets sent
    0 receive buffer errors
    0 send buffer errors
UdpLite:
TcpExt:
    2 TCP sockets finished time wait in fast timer
    113 delayed acks sent
    2 delayed acks further delayed because of locked socket
    Quick ack mode was activated 1 times
    1122 packet headers predicted
    170 acknowledgments not containing data payload received
    1247 predicted acknowledgments
    TCPTimeouts: 3
    TCPLossProbes: 1
    TCPBacklogCoalesce: 19
    TCPDSACKOldSent: 1
    TCPDSACKRecv: 1
    3 connections reset due to unexpected data
    4 connections reset due to early user close
    TCPDSACKIgnoredNoUndo: 1
    IPReversePathFilter: 1
    TCPRcvCoalesce: 10
    TCPAutoCorking: 2
    TCPSynRetrans: 3
    TCPOrigDataSent: 1688
    TCPKeepAlive: 7
    TCPDelivered: 1693
IpExt:
    InOctets: 43438
    OutOctets: 128792
    InNoECTPkts: 607
MPTcpExt:

[----] I, [2022-06-17T16:06:43.450533 #2062:2ac79ca3f94c]  INFO -- evm: Diagnostics: [Hosts file contents]
127.0.0.1        localhost localhost.localdomain localhost4 localhost4.localdomain4
::1              localhost localhost.localdomain localhost6 localhost6.localdomain6

[----] I, [2022-06-17T16:06:43.452947 #2062:2ac79ca3f94c]  INFO -- evm: Diagnostics: [FStab file contents]
# /etc/fstab
# Created by anaconda on Fri Feb 11 01:22:32 2022
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
              /dev/mapper/vg_system-lv_os                /  xfs   defaults 0 0
UUID=f51a58b8-5393-4658-b19f-6b20bec9d63a            /boot  xfs   defaults 0 0
            /dev/mapper/vg_system-lv_home            /home  xfs   defaults 0 0
             /dev/mapper/vg_system-lv_tmp             /tmp  xfs   defaults 0 0
             /dev/mapper/vg_system-lv_var             /var  xfs   defaults 0 0
                /dev/mapper/vg_data-lv_pg   /var/lib/pgsql  xfs   defaults 0 0
         /dev/mapper/vg_system-lv_var_log         /var/log  xfs   defaults 0 0
   /dev/mapper/vg_system-lv_var_log_audit   /var/log/audit  xfs   defaults 0 0
UUID=7d049523-dfe5-416d-a102-1454060b4e21 /var/www/miq_tmp  xfs rw,noatime 0 0
            /dev/mapper/vg_system-lv_swap             none swap   defaults 0 0

[----] I, [2022-06-17T16:07:23.445862 #2062:2ac79ca3f94c]  INFO -- evm: Diagnostics: [Installed RPMs]
[["GConf2", "3.2.6-22.el8"], ["NetworkManager", "1.36.0-0.7.el8"], ["NetworkManager-libnm", "1.36.0-0.7.el8"], ["NetworkManager-team", "1.36.0-0.7.el8"], ["NetworkManager-tui", "1.36.0-0.7.el8"], ["OpenIPMI", "2.0.31-3.el8"], ["OpenIPMI-libs", "2.0.31-3.el8"], ["PackageKit", "1.1.12-6.el8"], ["PackageKit-glib", "1.1.12-6.el8"], ["abattis-cantarell-fonts", "0.0.25-6.el8"], ["acl", "2.2.53-1.el8"], ["adcli", "0.8.2-12.el8"], ["alsa-lib", "1.2.6.1-1.el8"], ["ansible", "2.9.27-1.el8"], ["ansible-runner", "1.4.7-1.el8"], ["apr", "1.6.3-12.el8"], ["apr-util", "1.6.1-6.el8"], ["apr-util-bdb", "1.6.1-6.el8"], ["apr-util-openssl", "1.6.1-6.el8"], ["at", "3.1.20-11.el8"], ["atk", "2.28.1-1.el8"], ["attr", "2.4.48-3.el8"], ["audit", "3.0.7-1.el8"], ["audit-libs", "3.0.7-1.el8"], ["augeas-libs", "1.12.0-7.el8"], ["authselect", "1.2.2-3.el8"], ["authselect-compat", "1.2.2-3.el8"], ["authselect-libs", "1.2.2-3.el8"], ["autofs", "5.1.4-77.el8"], ["avahi-libs", "0.7-20.el8"], ["basesystem", "11-5.el8"], ["bash", "4.4.20-3.el8"], ["bc", "1.07.1-5.el8"], ["bind-export-libs", "9.11.36-2.el8"], ["bind-libs", "9.11.36-2.el8"], ["bind-libs-lite", "9.11.36-2.el8"], ["bind-license", "9.11.36-2.el8"], ["bind-utils", "9.11.36-2.el8"], ["binutils", "2.30-113.el8"], ["biosdevname", "0.7.3-2.el8"], ["brotli", "1.0.6-3.el8"], ["bzip2", "1.0.6-26.el8"], ["bzip2-libs", "1.0.6-26.el8"], ["c-ares", "1.13.0-6.el8"], ["ca-certificates", "2021.2.50-82.el8"], ["cairo", "1.15.12-3.el8"], ["cairo-gobject", "1.15.12-3.el8"], ["centos-gpg-keys", "8-4.el8"], ["centos-logos", "85.8-2.el8"], ["centos-logos-httpd", "85.8-2.el8"], ["centos-stream-release", "8.6-1.el8"], ["centos-stream-repos", "8-4.el8"], ["certmonger", "0.79.13-5.el8"], ["checkpolicy", "2.9-1.el8"], ["chkconfig", "1.19.1-1.el8"], ["chrony", "4.1-1.el8"], ["cifs-utils", "6.8-3.el8"], ["cjose", "0.6.1-2.module_el8.4.0+674+2c6c7264"], ["cloud-init", "21.1-13.el8"], ["cmake-filesystem", "3.20.2-4.el8"], ["cockpit", "261-1.el8"], ["cockpit-bridge", "261-1.el8"], ["cockpit-packagekit", "261-1.el8"], ["cockpit-system", "261-1.el8"], ["cockpit-ws", "261-1.el8"], ["copy-jdk-configs", "4.0-2.el8"], ["coreutils", "8.30-12.el8"], ["coreutils-common", "8.30-12.el8"], ["cpio", "2.12-11.el8"], ["cpp", "8.5.0-10.el8"], ["cracklib", "2.9.6-15.el8"], ["cracklib-dicts", "2.9.6-15.el8"], ["createrepo_c", "0.17.7-3.el8"], ["createrepo_c-libs", "0.17.7-3.el8"], ["cronie", "1.5.2-6.el8"], ["cronie-anacron", "1.5.2-6.el8"], ["crontabs", "1.11-17.20190603git.el8"], ["crypto-policies", "20211116-1.gitae470d6.el8"], ["crypto-policies-scripts", "20211116-1.gitae470d6.el8"], ["cryptsetup-libs", "2.3.7-1.el8"], ["cups-libs", "2.2.6-42.el8"], ["curl", "7.61.1-22.el8"], ["cyrus-sasl", "2.1.27-5.el8"], ["cyrus-sasl-gssapi", "2.1.27-5.el8"], ["cyrus-sasl-lib", "2.1.27-5.el8"], ["cyrus-sasl-plain", "2.1.27-5.el8"], ["dbus", "1.12.8-18.el8"], ["dbus-common", "1.12.8-18.el8"], ["dbus-daemon", "1.12.8-18.el8"], ["dbus-glib", "0.110-2.el8"], ["dbus-libs", "1.12.8-18.el8"], ["dbus-tools", "1.12.8-18.el8"], ["dejavu-fonts-common", "2.35-7.el8"], ["dejavu-sans-mono-fonts", "2.35-7.el8"], ["device-mapper", "1.02.181-3.el8"], ["device-mapper-event", "1.02.181-3.el8"], ["device-mapper-event-libs", "1.02.181-3.el8"], ["device-mapper-libs", "1.02.181-3.el8"], ["device-mapper-persistent-data", "0.9.0-6.el8"], ["dhcp-client", "4.3.6-47.el8.0.1"], ["dhcp-common", "4.3.6-47.el8.0.1"], ["dhcp-libs", "4.3.6-47.el8.0.1"], ["diffutils", "3.6-6.el8"], ["dmidecode", "3.3-1.el8"], ["dnf", "4.7.0-7.el8"], ["dnf-data", "4.7.0-7.el8"], ["dnf-plugins-core", "4.0.21-10.el8"], ["dracut", "049-201.git20220131.el8"], ["dracut-config-rescue", "049-201.git20220131.el8"], ["dracut-network", "049-201.git20220131.el8"], ["dracut-squash", "049-201.git20220131.el8"], ["drpm", "0.4.1-3.el8"], ["e2fsprogs", "1.45.6-3.el8"], ["e2fsprogs-libs", "1.45.6-3.el8"], ["ed", "1.14.2-4.el8"], ["elfutils-debuginfod-client", "0.186-1.el8"], ["elfutils-default-yama-scope", "0.186-1.el8"], ["elfutils-libelf", "0.186-1.el8"], ["elfutils-libs", "0.186-1.el8"], ["emacs-filesystem", "26.1-7.el8"], ["epel-next-release", "8-14.el8"], ["epel-release", "8-14.el8"], ["ethtool", "5.13-1.el8"], ["expat", "2.2.5-4.el8"], ["file", "5.33-20.el8"], ["file-libs", "5.33-20.el8"], ["filesystem", "3.8-6.el8"], ["findutils", "4.6.0-20.el8"], ["firewalld", "0.9.3-11.el8"], ["firewalld-filesystem", "0.9.3-11.el8"], ["fontconfig", "2.13.1-4.el8"], ["fontpackages-filesystem", "1.44-22.el8"], ["freeipmi", "1.6.8-1.el8"], ["freetype", "2.9.1-4.el8_3.1"], ["fribidi", "1.0.4-8.el8"], ["fstrm", "0.6.1-2.el8"], ["fuse-libs", "2.9.7-12.el8"], ["gawk", "4.2.1-2.el8"], ["gcc", "8.5.0-10.el8"], ["gdbm", "1.18-1.el8"], ["gdbm-libs", "1.18-1.el8"], ["gdisk", "1.0.3-8.el8"], ["gdk-pixbuf2", "2.36.12-5.el8"], ["gdk-pixbuf2-modules", "2.36.12-5.el8"], ["geolite2-city", "20180605-1.el8"], ["geolite2-country", "20180605-1.el8"], ["gettext", "0.19.8.1-17.el8"], ["gettext-libs", "0.19.8.1-17.el8"], ["giflib", "5.1.4-3.el8"], ["git", "2.31.1-2.el8"], ["git-core", "2.31.1-2.el8"], ["git-core-doc", "2.31.1-2.el8"], ["glib-networking", "2.56.1-1.1.el8"], ["glib2", "2.56.4-158.el8"], ["glibc", "2.28-189.el8"], ["glibc-common", "2.28-189.el8"], ["glibc-devel", "2.28-189.el8"], ["glibc-gconv-extra", "2.28-189.el8"], ["glibc-headers", "2.28-189.el8"], ["glibc-langpack-en", "2.28-189.el8"], ["glusterfs", "6.0-56.4.el8"], ["glusterfs-client-xlators", "6.0-56.4.el8"], ["glusterfs-fuse", "6.0-56.4.el8"], ["glusterfs-libs", "6.0-56.4.el8"], ["gmp", "6.1.2-10.el8"], ["gnupg2", "2.2.20-2.el8"], ["gnupg2-smime", "2.2.20-2.el8"], ["gnutls", "3.6.16-4.el8"], ["gobject-introspection", "1.56.1-1.el8"], ["gpg-pubkey", "8483c65d-5ccc5b19"], ["gpgme", "1.13.1-9.el8"], ["gpm-libs", "1.20.7-17.el8"], ["graphite2", "1.3.10-10.el8"], ["grep", "3.1-6.el8"], ["groff-base", "1.22.3-18.el8"], ["grub2-common", "2.02-106.el8"], ["grub2-pc", "2.02-106.el8"], ["grub2-pc-modules", "2.02-106.el8"], ["grub2-tools", "2.02-106.el8"], ["grub2-tools-extra", "2.02-106.el8"], ["grub2-tools-minimal", "2.02-106.el8"], ["grubby", "8.40-42.el8"], ["gsettings-desktop-schemas", "3.32.0-6.el8"], ["gssproxy", "0.8.0-20.el8"], ["gtk-update-icon-cache", "3.22.30-8.el8"], ["gtk2", "2.24.32-5.el8"], ["gzip", "1.9-12.el8"], ["hardlink", "1.3-6.el8"], ["harfbuzz", "1.7.5-3.el8"], ["hdparm", "9.54-4.el8"], ["hicolor-icon-theme", "0.17-2.el8"], ["hostname", "3.20-7.el8.0.1"], ["httpd", "2.4.37-43.module_el8.5.0+1022+b541f3b1"], ["httpd-filesystem", "2.4.37-43.module_el8.5.0+1022+b541f3b1"], ["httpd-tools", "2.4.37-43.module_el8.5.0+1022+b541f3b1"], ["hwdata", "0.314-8.11.el8"], ["ima-evm-utils", "1.3.2-12.el8"], ["info", "6.5-7.el8_5"], ["initscripts", "10.00.17-1.el8"], ["ipa-client", "4.9.8-2.module_el8.6.0+1054+cdb51b28"], ["ipa-client-common", "4.9.8-2.module_el8.6.0+1054+cdb51b28"], ["ipa-common", "4.9.8-2.module_el8.6.0+1054+cdb51b28"], ["ipa-selinux", "4.9.8-2.module_el8.6.0+1054+cdb51b28"], ["ipcalc", "0.2.4-4.el8"], ["ipmitool", "1.8.18-18.el8"], ["iproute", "5.15.0-2.el8"], ["iprutils", "2.4.19-1.el8"], ["ipset", "7.1-1.el8"], ["ipset-libs", "7.1-1.el8"], ["iptables", "1.8.4-22.el8"], ["iptables-ebtables", "1.8.4-22.el8"], ["iptables-libs", "1.8.4-22.el8"], ["iptstate", "2.2.6-6.el8"], ["iputils", "20180629-8.el8"], ["irqbalance", "1.4.0-6.el8"], ["isl", "0.16.1-6.el8"], ["jansson", "2.14-1.el8"], ["jasper-libs", "2.0.14-5.el8"], ["java-1.8.0-openjdk", "1.8.0.322.b06-2.el8_5"], ["java-1.8.0-openjdk-headless", "1.8.0.322.b06-2.el8_5"], ["javapackages-filesystem", "5.3.0-1.module_el8.0.0+11+5b8c10bd"], ["jbigkit-libs", "2.1-14.el8"], ["jq", "1.6-3.el8"], ["json-c", "0.13.1-3.el8"], ["json-glib", "1.4.4-1.el8"], ["kafka", "2.3.1-2"], ["kbd", "2.0.4-10.el8"], ["kbd-legacy", "2.0.4-10.el8"], ["kbd-misc", "2.0.4-10.el8"], ["kernel", "4.18.0-365.el8"], ["kernel-core", "4.18.0-365.el8"], ["kernel-headers", "4.18.0-358.el8"], ["kernel-modules", "4.18.0-365.el8"], ["kernel-tools", "4.18.0-365.el8"], ["kernel-tools-libs", "4.18.0-365.el8"], ["kexec-tools", "2.0.20-68.el8"], ["keyutils", "1.5.10-9.el8"], ["keyutils-libs", "1.5.10-9.el8"], ["keyutils-libs-devel", "1.5.10-9....
[----] I, [2022-06-17T16:07:25.051606 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding...
[----] I, [2022-06-17T16:07:25.248856 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqWidget...
[----] I, [2022-06-17T16:07:25.685200 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [chart_guest_os_information_any_os] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:26.936158 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [chart_hosts_summary_by_version] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.016612 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [Number of Nodes per CPU Cores Chart] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.097048 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [Pods per Ready Status Chart] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.173023 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [chart_server_availability] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.247180 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [chart_server_health] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.302194 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [chart_vendor_and_guest_os] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.356542 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [chart_virtual_infrastructure_platforms] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.409409 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [CPU Right Sizing Recommendations] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.464236 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [Nodes by Capacity] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.524103 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [Nodes By CPU Usage] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.579920 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [Nodes By Memory Usage] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.642327 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [Pods per Ready Status] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.698771 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [Projects By CPU Usage] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.753880 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [Projects By Memory Usage] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.829534 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [Projects by Number of Containers] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.900568 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [Projects by Number of Pods] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:27.955620 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [Recently Discovered Pods] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:28.015206 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [report_top_cpu_consumers_weekly] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:28.069265 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [report_top_memory_consumers_weekly] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:28.135140 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [report_top_storage_consumers] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:28.204552 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [rss_newest_hosts] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:28.271167 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [rss_newest_servers] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:28.337254 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [rss_newest_vms] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:28.406570 #2062:2ac79ca3f94c]  INFO -- evm: Widget: [tenant_quotas] file has been updated on disk, synchronizing with model
[----] I, [2022-06-17T16:07:28.446656 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqWidget... Complete in 3195.0934270000744ms
[----] I, [2022-06-17T16:07:28.448859 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqAction...
[----] I, [2022-06-17T16:07:28.968541 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqAction... Complete in 517.6011069999049ms
[----] I, [2022-06-17T16:07:28.970169 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqEventDefinitionSet...
[----] I, [2022-06-17T16:07:29.323356 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqEventDefinitionSet... Complete in 351.36702800002695ms
[----] I, [2022-06-17T16:07:29.328592 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqEventDefinition...
[----] I, [2022-06-17T16:07:37.013797 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqEventDefinition... Complete in 7682.814320000034ms
[----] I, [2022-06-17T16:07:37.016730 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqPolicySet...
[----] I, [2022-06-17T16:07:40.569826 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqPolicySet... Complete in 3550.6626249999726ms
[----] I, [2022-06-17T16:07:40.571656 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ChargebackRateDetailMeasure...
[----] I, [2022-06-17T16:07:40.803298 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ChargebackRateDetailMeasure... Complete in 229.43006000002697ms
[----] I, [2022-06-17T16:07:40.804881 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ChargeableField...
[----] I, [2022-06-17T16:07:41.039125 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ChargeableField... Complete in 232.37537900001826ms
[----] I, [2022-06-17T16:07:41.042000 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding Currency...
[----] I, [2022-06-17T16:07:41.300699 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(Currency.seed) Creating [XBA] with symbols=[]
[----] I, [2022-06-17T16:07:41.321747 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(Currency.seed) Creating [XBB] with symbols=[]
[----] I, [2022-06-17T16:07:41.330166 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(Currency.seed) Creating [XBC] with symbols=[]
[----] I, [2022-06-17T16:07:41.338108 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(Currency.seed) Creating [XBD] with symbols=[]
[----] I, [2022-06-17T16:07:41.347777 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(Currency.seed) Creating [XTS] with symbols=[]
[----] I, [2022-06-17T16:07:41.356941 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding Currency... Complete in 312.8577500000347ms
[----] I, [2022-06-17T16:07:41.358597 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ChargebackRate...
[----] I, [2022-06-17T16:07:42.426103 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ChargebackRate... Complete in 1065.5555049999066ms
[----] I, [2022-06-17T16:07:42.427843 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding BlacklistedEvent...
[----] I, [2022-06-17T16:07:43.831726 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding BlacklistedEvent... Complete in 1401.835842999958ms
[----] I, [2022-06-17T16:07:43.833644 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding Classification...
[----] I, [2022-06-17T16:07:44.785946 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding Classification... Complete in 950.250321999988ms
[----] I, [2022-06-17T16:07:44.787913 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding CustomizationTemplate...
[----] I, [2022-06-17T16:07:46.011106 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding CustomizationTemplate... Complete in 1220.5972069999689ms
[----] I, [2022-06-17T16:07:46.017144 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding Dialog...
[----] I, [2022-06-17T16:07:51.283850 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding Dialog... Complete in 5263.661014000036ms
[----] I, [2022-06-17T16:07:51.286360 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ManageIQ::Providers::EmbeddedAnsible...
[----] I, [2022-06-17T16:08:10.897894 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ManageIQ::Providers::EmbeddedAnsible... Complete in 19606.796605ms
[----] I, [2022-06-17T16:08:10.899646 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqAlert...
[----] I, [2022-06-17T16:08:11.523117 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqAlert... Complete in 621.1779330000127ms
[----] I, [2022-06-17T16:08:11.525564 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqAlertSet...
[----] I, [2022-06-17T16:08:12.382001 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqExpression.build_relats) Building relationship tree for: [ => EmsClusterPerformance]...
[----] I, [2022-06-17T16:08:13.168783 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqExpression.build_relats) Building relationship tree for: [ => EmsCluster]...
[----] I, [2022-06-17T16:08:20.308873 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqAlertSet... Complete in 8779.334972000015ms
[----] I, [2022-06-17T16:08:20.310845 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqDialog...
[----] I, [2022-06-17T16:08:20.740712 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqDialog... Complete in 427.83151599996927ms
[----] I, [2022-06-17T16:08:20.743368 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqSearch...
[----] I, [2022-06-17T16:08:21.628363 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqSearch... Complete in 882.6480789999778ms
[----] I, [2022-06-17T16:08:21.631431 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqShortcut...
[----] I, [2022-06-17T16:08:22.027392 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqShortcut... Complete in 393.5916970000335ms
[----] I, [2022-06-17T16:08:22.032431 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqWidgetSet...
[----] I, [2022-06-17T16:08:22.272530 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqWidgetSet... Complete in 238.04976899998564ms
[----] I, [2022-06-17T16:08:22.274457 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding NotificationType...
[----] I, [2022-06-17T16:08:23.290209 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding NotificationType... Complete in 1013.9304250000123ms
[----] I, [2022-06-17T16:08:23.292239 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding PxeImageType...
[----] I, [2022-06-17T16:08:23.328879 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding PxeImageType... Complete in 34.14120399997955ms
[----] I, [2022-06-17T16:08:23.330826 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ScanItem...
[----] I, [2022-06-17T16:08:24.365658 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ScanItem... Complete in 1032.7605930000345ms
[----] I, [2022-06-17T16:08:24.367777 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding TimeProfile...
[----] I, [2022-06-17T16:08:24.586060 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding TimeProfile... Complete in 216.22315500007971ms
[----] I, [2022-06-17T16:08:24.588617 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqAeDatastore...
[----] I, [2022-06-17T16:08:24.623163 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqAeDatastore.seed) Resetting domain priorities at startup...
[----] I, [2022-06-17T16:08:24.676603 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding MiqAeDatastore... Complete in 85.84762100008447ms
[----] I, [2022-06-17T16:08:24.678969 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ManageIQ::Providers::Vmware...
[----] I, [2022-06-17T16:08:24.958488 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding ManageIQ::Providers::Vmware... Complete in 277.5483120000217ms
[----] I, [2022-06-17T16:08:25.002632 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmDatabase.seed_classes) Seeding... Complete in 59947.86744999999ms
[----] I, [2022-06-17T16:08:26.828135 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqMemcached::Control.stop) stopped memcached
[----] E, [2022-06-17T16:08:26.991964 #2062:2ac79ca3f94c] ERROR -- evm: AwesomeSpawn: killall exit code: 1
[----] E, [2022-06-17T16:08:26.994109 #2062:2ac79ca3f94c] ERROR -- evm: AwesomeSpawn: memcached: no process found

[----] I, [2022-06-17T16:08:27.239863 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqMemcached::Control.start) started memcached with options: {:port=>"11211", :options=>"-l 127.0.0.1 -I 1M"}
[----] I, [2022-06-17T16:08:28.047063 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer.start_memcached) Status: Redirecting to /bin/systemctl status memcached.service
● memcached.service - memcached daemon
   Loaded: loaded (/usr/lib/systemd/system/memcached.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2022-06-17 16:08:27 EDT; 788ms ago
 Main PID: 2147 (memcached)
    Tasks: 10 (limit: 94586)
   Memory: 3.7M
   CGroup: /system.slice/memcached.service
           └─2147 /usr/bin/memcached -p 11211 -u memcached -m 64 -c 1024 -l 127.0.0.1 -I 1M

Jun 17 16:08:27 localhost.localdomain systemd[1]: Started memcached daemon.
[----] I, [2022-06-17T16:08:36.684114 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000075],  id: [], Zone: [default], Role: [automate], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1000000000001, :attrs=>{:event_type=>"evm_server_start", "MiqEvent::miq_event"=>1000000000037, :miq_event_id=>1000000000037, "EventStream::event_stream"=>1000000000037, :event_stream_id=>1000000000037}, :instance_name=>"Event", :user_id=>1000000000001, :miq_group_id=>1000000000002, :tenant_id=>1000000000001, :automate_message=>nil}]
[----] I, [2022-06-17T16:08:36.685692 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#start_server) Server starting...
[----] I, [2022-06-17T16:08:36.929102 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#deactivate) Deactivating Role <automate> on Server <EVM>
[----] I, [2022-06-17T16:08:36.972692 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#deactivate) Deactivating Role <database_operations> on Server <EVM>
[----] I, [2022-06-17T16:08:37.052743 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#deactivate) Deactivating Role <ems_inventory> on Server <EVM>
[----] I, [2022-06-17T16:08:37.112771 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#deactivate) Deactivating Role <ems_operations> on Server <EVM>
[----] I, [2022-06-17T16:08:37.196125 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#deactivate) Deactivating Role <event> on Server <EVM>
[----] I, [2022-06-17T16:08:37.285745 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#deactivate) Deactivating Role <reporting> on Server <EVM>
[----] I, [2022-06-17T16:08:37.362882 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#deactivate) Deactivating Role <scheduler> on Server <EVM>
[----] I, [2022-06-17T16:08:37.445017 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#deactivate) Deactivating Role <smartstate> on Server <EVM>
[----] I, [2022-06-17T16:08:37.531174 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#deactivate) Deactivating Role <user_interface> on Server <EVM>
[----] I, [2022-06-17T16:08:37.610825 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#deactivate) Deactivating Role <remote_console> on Server <EVM>
[----] I, [2022-06-17T16:08:37.669735 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#deactivate) Deactivating Role <web_services> on Server <EVM>
[----] I, [2022-06-17T16:08:37.814137 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#make_master_server) Master server has not been set.  Attempting takeover as new master server, EVM.
[----] I, [2022-06-17T16:08:37.925367 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#make_master_server) This server EVM is now set as the master server, last_master: 
[----] I, [2022-06-17T16:08:38.003255 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000076],  id: [], Zone: [], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_server_is_master", {:event_details=>"MiqServer [EVM] with ID: [1000000000001] has taken over master"}]
[----] I, [2022-06-17T16:08:38.946008 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#activate) Activating Role <ems_inventory> on Server <EVM>
[----] I, [2022-06-17T16:08:38.982122 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#activate) Activating Role <ems_operations> on Server <EVM>
[----] I, [2022-06-17T16:08:39.014199 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#activate) Activating Role <event> on Server <EVM>
[----] I, [2022-06-17T16:08:39.046815 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#activate) Activating Role <smartstate> on Server <EVM>
[----] I, [2022-06-17T16:08:39.304018 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#activate) Activating Role <automate> on Server <EVM>
[----] I, [2022-06-17T16:08:39.345201 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#activate) Activating Role <database_operations> on Server <EVM>
[----] I, [2022-06-17T16:08:39.379244 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#activate) Activating Role <reporting> on Server <EVM>
[----] I, [2022-06-17T16:08:39.411237 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#activate) Activating Role <scheduler> on Server <EVM>
[----] I, [2022-06-17T16:08:39.445539 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#activate) Activating Role <user_interface> on Server <EVM>
[----] I, [2022-06-17T16:08:39.480366 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#activate) Activating Role <remote_console> on Server <EVM>
[----] I, [2022-06-17T16:08:39.515386 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(AssignedServerRole#activate) Activating Role <web_services> on Server <EVM>
[----] I, [2022-06-17T16:08:41.533034 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer.log_managed_entities) Region: [1], name: [Region 1], Under Management: VMs: [0], Hosts: [0], Sockets: [0.0]
[----] I, [2022-06-17T16:08:41.576214 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer.log_managed_entities) Region: [1], name: [Region 1], Not Under Management: VMs: [0], Hosts: [0], Sockets: [0.0]
[----] I, [2022-06-17T16:08:41.578523 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer.write_systemd_unit_files) Writing Systemd unit files...
[----] I, [2022-06-17T16:08:44.118165 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer.write_systemd_unit_files) Writing Systemd unit files...Complete
[----] I, [2022-06-17T16:08:44.119725 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer.clean_all_workers) Cleaning up all workers...
[----] I, [2022-06-17T16:08:44.187795 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer.clean_all_workers) Cleaning up all workers...Complete
[----] I, [2022-06-17T16:08:44.189631 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer.clean_dequeued_messages) Cleaning up dequeued messages...
[----] I, [2022-06-17T16:08:44.206786 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer.clean_dequeued_messages) Cleaning up dequeued messages...Complete
[----] I, [2022-06-17T16:08:44.208978 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer.purge_report_results) Purging adhoc report results...
[----] I, [2022-06-17T16:08:44.447966 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer.purge_report_results) Purging adhoc report results...Complete
[----] I, [2022-06-17T16:08:44.469102 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#delete_active_log_collections_queue) Previous cleanup is still running, skipping...
[----] I, [2022-06-17T16:08:45.491903 #2062:2ac79ca3f94c]  INFO -- evm: 
---
:enforce_resource_constraints: false
:kill_algorithm:
  :name: used_swap_percent_gt_value
  :value: 80
:miq_server_time_threshold: 120
:nice_delta: 1
:poll: 2
:start_algorithm:
  :name: used_swap_percent_lt_value
  :value: 60
:sync_interval: 1800
:wait_for_started_timeout: 600
[----] I, [2022-06-17T16:08:56.531288 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqPriorityWorker.sync_workers) Workers are being synchronized: Current #: [0], Desired #: [2]
[----] I, [2022-06-17T16:08:57.474063 #2062:2ac79ca3f94c]  INFO -- evm: 
---
:enforce_resource_constraints: false
:kill_algorithm:
  :name: used_swap_percent_gt_value
  :value: 80
:miq_server_time_threshold: 120
:nice_delta: 1
:poll: 2
:start_algorithm:
  :name: used_swap_percent_lt_value
  :value: 60
:sync_interval: 1800
:wait_for_started_timeout: 600
[----] I, [2022-06-17T16:08:59.173284 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000077],  id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_worker_start", {:event_details=>"Worker started: ID [1000000000056], PID [], GUID [786c32af-ff44-4f68-8555-ced8bd99f8c2]", :type=>"MiqPriorityWorker"}]
[----] I, [2022-06-17T16:08:59.180029 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqPriorityWorker#start) Worker started: ID [1000000000056], PID [], GUID [786c32af-ff44-4f68-8555-ced8bd99f8c2]
[----] I, [2022-06-17T16:08:59.638354 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000078],  id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_worker_start", {:event_details=>"Worker started: ID [1000000000057], PID [], GUID [57f5e49a-2355-4c38-87c5-7990961068ee]", :type=>"MiqPriorityWorker"}]
[----] I, [2022-06-17T16:08:59.639899 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqPriorityWorker#start) Worker started: ID [1000000000057], PID [], GUID [57f5e49a-2355-4c38-87c5-7990961068ee]
[----] I, [2022-06-17T16:08:59.739219 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqEventHandler.sync_workers) Workers are being synchronized: Current #: [0], Desired #: [1]
[----] I, [2022-06-17T16:09:00.779603 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000079],  id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_worker_start", {:event_details=>"Worker started: ID [1000000000058], PID [], GUID [615d0a37-4713-41b5-b0cb-72c28b352d86]", :type=>"MiqEventHandler"}]
[----] I, [2022-06-17T16:09:00.781389 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqEventHandler#start) Worker started: ID [1000000000058], PID [], GUID [615d0a37-4713-41b5-b0cb-72c28b352d86]
[----] I, [2022-06-17T16:09:00.855207 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqGenericWorker.sync_workers) Workers are being synchronized: Current #: [0], Desired #: [2]
[----] I, [2022-06-17T16:09:02.387845 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000080],  id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_worker_start", {:event_details=>"Worker started: ID [1000000000059], PID [], GUID [672cc192-f1fe-496f-a46f-6be105f31fab]", :type=>"MiqGenericWorker"}]
[----] I, [2022-06-17T16:09:02.389643 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqGenericWorker#start) Worker started: ID [1000000000059], PID [], GUID [672cc192-f1fe-496f-a46f-6be105f31fab]
[----] I, [2022-06-17T16:09:03.299057 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000081],  id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_worker_start", {:event_details=>"Worker started: ID [1000000000060], PID [], GUID [19cf6900-4f24-4654-8f46-aad09cf51665]", :type=>"MiqGenericWorker"}]
[----] I, [2022-06-17T16:09:03.300488 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqGenericWorker#start) Worker started: ID [1000000000060], PID [], GUID [19cf6900-4f24-4654-8f46-aad09cf51665]
[----] I, [2022-06-17T16:09:03.627032 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqReportingWorker.sync_workers) Workers are being synchronized: Current #: [0], Desired #: [2]
[----] I, [2022-06-17T16:09:06.095409 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000082],  id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_worker_start", {:event_details=>"Worker started: ID [1000000000061], PID [], GUID [f2e78240-f27f-42c0-b8d9-ca3498a93af6]", :type=>"MiqReportingWorker"}]
[----] I, [2022-06-17T16:09:06.097484 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqReportingWorker#start) Worker started: ID [1000000000061], PID [], GUID [f2e78240-f27f-42c0-b8d9-ca3498a93af6]
[----] I, [2022-06-17T16:09:07.264197 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000083],  id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_worker_start", {:event_details=>"Worker started: ID [1000000000062], PID [], GUID [0075ca28-2600-4f52-9d7b-5fbf13070228]", :type=>"MiqReportingWorker"}]
[----] I, [2022-06-17T16:09:07.265682 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqReportingWorker#start) Worker started: ID [1000000000062], PID [], GUID [0075ca28-2600-4f52-9d7b-5fbf13070228]
[----] I, [2022-06-17T16:09:07.610943 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqRemoteConsoleWorker.sync_workers) Workers are being synchronized: Current #: [0], Desired #: [1]
[----] I, [2022-06-17T16:09:07.616360 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqRemoteConsoleWorker.sync_workers) Reserved port=5000, Current ports in use: []
[----] I, [2022-06-17T16:09:11.409744 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000084],  id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_worker_start", {:event_details=>"Worker started: ID [1000000000063], PID [], GUID [1a612deb-7448-4ddc-9e70-f64f0469e36a]", :type=>"MiqRemoteConsoleWorker"}]
[----] I, [2022-06-17T16:09:11.411588 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqRemoteConsoleWorker#start) Worker started: ID [1000000000063], PID [], GUID [1a612deb-7448-4ddc-9e70-f64f0469e36a]
[----] I, [2022-06-17T16:09:11.591214 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqScheduleWorker.sync_workers) Workers are being synchronized: Current #: [0], Desired #: [1]
[----] I, [2022-06-17T16:09:15.454020 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000085],  id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_worker_start", {:event_details=>"Worker started: ID [1000000000064], PID [], GUID [941289a1-f8dc-47ae-9d13-0dd4ee3df810]", :type=>"MiqScheduleWorker"}]
[----] I, [2022-06-17T16:09:15.456407 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqScheduleWorker#start) Worker started: ID [1000000000064], PID [], GUID [941289a1-f8dc-47ae-9d13-0dd4ee3df810]
[----] I, [2022-06-17T16:09:15.816864 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqUiWorker.sync_workers) Workers are being synchronized: Current #: [0], Desired #: [1]
[----] I, [2022-06-17T16:09:15.822426 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqUiWorker.sync_workers) Reserved port=3000, Current ports in use: []
[----] I, [2022-06-17T16:09:19.539956 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000086],  id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_worker_start", {:event_details=>"Worker started: ID [1000000000065], PID [], GUID [e24940d5-c52d-43a1-ba57-266454f30dfb]", :type=>"MiqUiWorker"}]
[----] I, [2022-06-17T16:09:19.543170 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqUiWorker#start) Worker started: ID [1000000000065], PID [], GUID [e24940d5-c52d-43a1-ba57-266454f30dfb]
[----] I, [2022-06-17T16:09:20.071006 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqWebServiceWorker.sync_workers) Workers are being synchronized: Current #: [0], Desired #: [1]
[----] I, [2022-06-17T16:09:20.114268 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqWebServiceWorker.sync_workers) Reserved port=4000, Current ports in use: []
[----] I, [2022-06-17T16:09:24.372011 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqQueue.put) Message id: [1000000000087],  id: [], Zone: [default], Role: [], Server: [], MiqTask id: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1000000000001], "evm_worker_start", {:event_details=>"Worker started: ID [1000000000066], PID [], GUID [6b47c5e7-a561-4dc9-9394-0e393c4abfe0]", :type=>"MiqWebServiceWorker"}]
[----] I, [2022-06-17T16:09:24.373839 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqWebServiceWorker#start) Worker started: ID [1000000000066], PID [], GUID [6b47c5e7-a561-4dc9-9394-0e393c4abfe0]
[----] I, [2022-06-17T16:09:27.569514 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#wait_for_started_workers) Waiting for the following workers to start: MiqEventHandler (1), MiqGenericWorker (2), MiqPriorityWorker (2), MiqRemoteConsoleWorker (1), MiqReportingWorker (2), MiqScheduleWorker (1), MiqUiWorker (1), MiqWebServiceWorker (1)
[----] I, [2022-06-17T16:16:32.791124 #2413:2b13f9233970]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:32.798205 #2413:2b13f9233970]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:33.564607 #2406:2b0f5937d960]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:33.570871 #2406:2b0f5937d960]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:36.321772 #2413:2b13f9233970]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:16:36.886712 #2406:2b0f5937d960]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:16:38.208903 #2436:2ab0411d795c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:38.268321 #2436:2ab0411d795c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:41.404812 #2436:2ab0411d795c]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:16:44.253428 #2469:2ab7696e396c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:44.297604 #2469:2ab7696e396c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:44.315696 #2538:2b015853396c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:44.329408 #2538:2b015853396c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:47.031827 #2469:2ab7696e396c]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:16:47.055296 #2538:2b015853396c]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:16:48.489904 #2500:2b0df6c71968]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:48.579600 #2500:2b0df6c71968]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:50.515626 #2507:2b2287b21964]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:50.586531 #2507:2b2287b21964]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:51.289451 #2455:2b00e4cb595c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:51.379892 #2455:2b00e4cb595c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:16:52.022542 #2500:2b0df6c71968]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:16:53.919465 #2507:2b2287b21964]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:16:54.439215 #2455:2b00e4cb595c]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:16:56.183113 #2406:2b0f5937d960]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2406, ENV['EVMSERVER']: 
[----] I, [2022-06-17T16:16:56.941048 #2413:2b13f9233970]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2413, ENV['EVMSERVER']: 
[----] I, [2022-06-17T16:17:02.113349 #2436:2ab0411d795c]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2436, ENV['EVMSERVER']: 
[----] I, [2022-06-17T16:17:05.295194 #2552:2b07c334795c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:17:05.367214 #2552:2b07c334795c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:17:06.302645 #2469:2ab7696e396c]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2469, ENV['EVMSERVER']: 
[----] I, [2022-06-17T16:17:06.768186 #2538:2b015853396c]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2538, ENV['EVMSERVER']: 
[----] I, [2022-06-17T16:17:08.563042 #2552:2b07c334795c]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:17:11.888306 #2500:2b0df6c71968]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2500, ENV['EVMSERVER']: 
[----] I, [2022-06-17T16:17:13.948361 #2455:2b00e4cb595c]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2455, ENV['EVMSERVER']: 
[----] I, [2022-06-17T16:17:14.084032 #2507:2b2287b21964]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2507, ENV['EVMSERVER']: 
[----] I, [2022-06-17T16:17:25.872091 #2552:2b07c334795c]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2552, ENV['EVMSERVER']: 
[----] I, [2022-06-17T16:17:28.705802 #2569:2af39383195c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:17:28.796089 #2569:2af39383195c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:17:29.844942 #2595:2ad03344b94c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
[----] I, [2022-06-17T16:17:29.850428 #2595:2ad03344b94c]  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
[----] I, [2022-06-17T16:17:32.278045 #2569:2af39383195c]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:17:32.820061 #2595:2ad03344b94c]  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
[----] I, [2022-06-17T16:17:50.861560 #2595:2ad03344b94c]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2595, ENV['EVMSERVER']: 
[----] I, [2022-06-17T16:17:51.163781 #2569:2af39383195c]  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2569, ENV['EVMSERVER']: 
[----] W, [2022-06-17T16:19:33.097642 #2062:2ac79ca3f94c]  WARN -- evm: MIQ(MiqServer#wait_for_started_workers) After waiting 600 seconds, no longer waiting for the following workers to start:
[----] W, [2022-06-17T16:19:33.109158 #2062:2ac79ca3f94c]  WARN -- evm: Worker type: MiqWebServiceWorker, pid: , guid: 6b47c5e7-a561-4dc9-9394-0e393c4abfe0, status: creating
[----] W, [2022-06-17T16:19:33.111892 #2062:2ac79ca3f94c]  WARN -- evm: Worker type: MiqUiWorker, pid: , guid: e24940d5-c52d-43a1-ba57-266454f30dfb, status: creating
[----] W, [2022-06-17T16:19:33.114321 #2062:2ac79ca3f94c]  WARN -- evm: Worker type: MiqScheduleWorker, pid: , guid: 941289a1-f8dc-47ae-9d13-0dd4ee3df810, status: creating
[----] W, [2022-06-17T16:19:33.116483 #2062:2ac79ca3f94c]  WARN -- evm: Worker type: MiqRemoteConsoleWorker, pid: , guid: 1a612deb-7448-4ddc-9e70-f64f0469e36a, status: creating
[----] W, [2022-06-17T16:19:33.118774 #2062:2ac79ca3f94c]  WARN -- evm: Worker type: MiqReportingWorker, pid: , guid: 0075ca28-2600-4f52-9d7b-5fbf13070228, status: creating
[----] W, [2022-06-17T16:19:33.120547 #2062:2ac79ca3f94c]  WARN -- evm: Worker type: MiqReportingWorker, pid: , guid: f2e78240-f27f-42c0-b8d9-ca3498a93af6, status: creating
[----] W, [2022-06-17T16:19:33.122774 #2062:2ac79ca3f94c]  WARN -- evm: Worker type: MiqGenericWorker, pid: , guid: 19cf6900-4f24-4654-8f46-aad09cf51665, status: creating
[----] W, [2022-06-17T16:19:33.125181 #2062:2ac79ca3f94c]  WARN -- evm: Worker type: MiqGenericWorker, pid: , guid: 672cc192-f1fe-496f-a46f-6be105f31fab, status: creating
[----] W, [2022-06-17T16:19:33.127697 #2062:2ac79ca3f94c]  WARN -- evm: Worker type: MiqEventHandler, pid: , guid: 615d0a37-4713-41b5-b0cb-72c28b352d86, status: creating
[----] W, [2022-06-17T16:19:33.130151 #2062:2ac79ca3f94c]  WARN -- evm: Worker type: MiqPriorityWorker, pid: , guid: 57f5e49a-2355-4c38-87c5-7990961068ee, status: creating
[----] W, [2022-06-17T16:19:33.132110 #2062:2ac79ca3f94c]  WARN -- evm: Worker type: MiqPriorityWorker, pid: , guid: 786c32af-ff44-4f68-8555-ced8bd99f8c2, status: creating
[----] I, [2022-06-17T16:19:33.256371 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#start_server) Server starting...Complete
[----] I, [2022-06-17T16:19:34.278758 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2022-06-17 20:19:34 UTC]...
[----] I, [2022-06-17T16:19:34.338985 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2022-06-17 20:19:34 UTC]...Complete
[----] I, [2022-06-17T16:19:34.981301 #2062:2ac79ca3f94c]  INFO -- evm: MiqServer: local=Y, master=Y, status= started, id=1000000000001, pid=02062, guid=e246d21e-61f4-4885-8072-f8d50e66b0f6, name=EVM, zone=default, hostname=, ipaddress=10.0.2.15, version=morphy-1, build=20220210224352_1e24154, active roles=automate:database_operations:database_owner:ems_inventory:ems_operations:event:remote_console:reporting:scheduler:smartstate:user_interface:web_services
[----] I, [2022-06-17T16:19:38.417685 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#cleanup_failed_systemd_services) Disabling failed unit files: [reporting@0075ca28-2600-4f52-9d7b-5fbf13070228.service, web_service@6b47c5e7-a561-4dc9-9394-0e393c4abfe0.service, generic@19cf6900-4f24-4654-8f46-aad09cf51665.service, reporting@f2e78240-f27f-42c0-b8d9-ca3498a93af6.service, priority@786c32af-ff44-4f68-8555-ced8bd99f8c2.service, remote_console@1a612deb-7448-4ddc-9e70-f64f0469e36a.service, schedule@941289a1-f8dc-47ae-9d13-0dd4ee3df810.service, ui@e24940d5-c52d-43a1-ba57-266454f30dfb.service, priority@57f5e49a-2355-4c38-87c5-7990961068ee.service, event_handler@615d0a37-4713-41b5-b0cb-72c28b352d86.service, generic@672cc192-f1fe-496f-a46f-6be105f31fab.service]
[----] I, [2022-06-17T16:19:38.939717 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#monitor) Server Monitoring Complete - Timings: {:heartbeat=>0.06293988227844238, :server_dequeue=>0.047025442123413086, :server_monitor=>0.545616626739502, :log_active_servers=>0.04707217216491699, :worker_monitor=>3.9509098529815674, :worker_dequeue=>0.0005414485931396484, :total_time=>4.6672210693359375}
[----] I, [2022-06-17T16:19:57.872406 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#monitor) Server Monitoring Complete - Timings: {:server_dequeue=>0.042305707931518555, :worker_monitor=>3.602417469024658, :worker_dequeue=>0.0005066394805908203, :total_time=>3.6477508544921875}
[----] I, [2022-06-17T16:20:08.015637 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2022-06-17 20:20:08 UTC]...
[----] I, [2022-06-17T16:20:08.127384 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2022-06-17 20:20:08 UTC]...Complete
[----] I, [2022-06-17T16:20:16.471057 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#monitor) Server Monitoring Complete - Timings: {:server_dequeue=>0.03617262840270996, :worker_monitor=>3.2173049449920654, :worker_dequeue=>0.00031876564025878906, :total_time=>3.255629301071167}
[----] I, [2022-06-17T16:20:35.264742 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#monitor) Server Monitoring Complete - Timings: {:server_dequeue=>0.04400062561035156, :worker_monitor=>3.430860996246338, :worker_dequeue=>0.00032830238342285156, :total_time=>3.4770052433013916}
[----] I, [2022-06-17T16:20:40.326749 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2022-06-17 20:20:40 UTC]...
[----] I, [2022-06-17T16:20:40.565559 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2022-06-17 20:20:40 UTC]...Complete
[----] I, [2022-06-17T16:20:54.679005 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#monitor) Server Monitoring Complete - Timings: {:server_dequeue=>0.03753376007080078, :worker_monitor=>3.3217434883117676, :worker_dequeue=>0.0003230571746826172, :total_time=>3.361778974533081}
[----] I, [2022-06-17T16:21:13.055384 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#monitor) Server Monitoring Complete - Timings: {:server_dequeue=>0.04693794250488281, :worker_monitor=>3.0119478702545166, :worker_dequeue=>0.00032782554626464844, :total_time=>3.0618155002593994}
[----] I, [2022-06-17T16:21:18.123284 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2022-06-17 20:21:18 UTC]...
[----] I, [2022-06-17T16:21:18.198774 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2022-06-17 20:21:18 UTC]...Complete
[----] I, [2022-06-17T16:21:31.557746 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#monitor) Server Monitoring Complete - Timings: {:server_dequeue=>0.04521751403808594, :worker_monitor=>3.0933029651641846, :worker_dequeue=>0.0003161430358886719, :total_time=>3.141237258911133}
[----] I, [2022-06-17T16:21:50.548018 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#monitor) Server Monitoring Complete - Timings: {:server_dequeue=>0.05675053596496582, :worker_monitor=>3.119413375854492, :worker_dequeue=>0.0002949237823486328, :total_time=>3.1785032749176025}
[----] I, [2022-06-17T16:21:55.618475 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2022-06-17 20:21:55 UTC]...
[----] I, [2022-06-17T16:21:55.683491 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(MiqServer#heartbeat) Heartbeat [2022-06-17 20:21:55 UTC]...Complete
[----] I, [2022-06-17T16:22:09.287307 #2062:2ac79ca3f94c]  INFO -- evm: MIQ(EvmServer#monitor) Server Monitoring Complete - Timings: {:server_dequeue=>0.056273698806762695, :worker_monitor=>3.299022912979126, :worker_dequeue=>0.00031447410583496094, :total_time=>3.3583309650421143}
kTipSSIoYv commented 2 years ago

@agrare Can you let me know if you find anything odd? I'm still trying to troubleshoot. I have no clue what the issue is at the moment.

remo233 commented 2 years ago

Hi @kTipSSIoYv can you share your resource details for manageiq server.

kTipSSIoYv commented 2 years ago

15GB of memory and using the qemu image disk as the hda.

qemu-system-x86_64 -boot d -hda manageiq-openstack-morphy-1.qc2 -m 15360 -smp cpus=2,cores=2 -netdev user,id=n1,hostfwd=tcp::2225-:22,hostfwd=tcp::9000-:9090,hostfwd=tcp::9843-:443,hostfwd=tcp::8085-:80 -device e1000,netdev=n1 -no-hpet -rtc driftfix=slew -global kvm-pit.lost_tick_policy=discard
agrare commented 2 years ago

@kTipSSIoYv can you try running systemctl status generic@19cf6900-4f24-4654-8f46-aad09cf51665.service and journald -u generic@19cf6900-4f24-4654-8f46-aad09cf51665.service ? It seems the workers aren't starting up properly

kTipSSIoYv commented 2 years ago

It seems it wasn't starting due to timeout. I increased the timeout but I see a different error when I try to start manually.

[root@localhost ~]# systemctl status generic@19cf6900-4f24-4654-8f46-aad09cf51665.service
● generic@19cf6900-4f24-4654-8f46-aad09cf51665.service
   Loaded: loaded (/etc/systemd/system/generic@.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api: Dynamic Configuration
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api:   token_ttl               : 10.minutes
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api:   authentication_timeout  : 30.seconds
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api:   max_results_per_page    : 1000
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 4146, ENV['EVMSERVER']:
Jun 19 10:42:31 localhost.localdomain bash[4146]: /opt/manageiq/manageiq-gemset/gems/activerecord-6.0.4.4/lib/active_record/core.rb:211:in `find_by!': Couldn't find MiqGenericWorker (ActiveRecord::RecordNotFound)
Jun 19 10:42:31 localhost.localdomain bash[4146]:         from lib/workers/bin/run_single_worker.rb:115:in `<main>'
Jun 19 10:42:31 localhost.localdomain systemd[1]: generic@19cf6900-4f24-4654-8f46-aad09cf51665.service: Main process exited, code=exited, status=1/FAILURE
Jun 19 10:42:31 localhost.localdomain systemd[1]: generic@19cf6900-4f24-4654-8f46-aad09cf51665.service: Failed with result 'exit-code'.
Jun 19 10:42:31 localhost.localdomain systemd[1]: Failed to start generic@19cf6900-4f24-4654-8f46-aad09cf51665.service.

journalctl log:


[root@localhost ~]# journalctl -u generic@19cf6900-4f24-4654-8f46-aad09cf51665.service
Journal file /var/log/journal/f15a3d1342104e59aa7f63a7a765d920/system@0005e1a7f7bcd28a-6892ab3f1e1d0c96.journal~ is truncated, ignoring file.
-- Logs begin at Fri 2022-06-17 13:19:31 EDT, end at Sun 2022-06-19 10:43:19 EDT. --
Jun 17 16:09:03 localhost.localdomain systemd[1]: Starting generic@19cf6900-4f24-4654-8f46-aad09cf51665.service...
Jun 17 16:14:52 localhost.localdomain bash[2469]: ** Booting MiqGenericWorker with PID: 2469 and options: {:heartbeat=>true, :guid=>"19cf6900-4f24-4654-8f46-aad09cf51665"}...
Jun 17 16:16:44 localhost.localdomain manageiq[2469]:  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
Jun 17 16:16:44 localhost.localdomain manageiq[2469]:  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
Jun 17 16:16:47 localhost.localdomain manageiq[2469]:  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
Jun 17 16:17:05 localhost.localdomain manageiq[2469]:  INFO -- api: Initializing Environment for API
Jun 17 16:17:05 localhost.localdomain manageiq[2469]:  INFO -- api:
Jun 17 16:17:05 localhost.localdomain manageiq[2469]:  INFO -- api: Static Configuration
Jun 17 16:17:06 localhost.localdomain manageiq[2469]:  INFO -- api:   module                  : api
Jun 17 16:17:06 localhost.localdomain manageiq[2469]:  INFO -- api:   name                    : API
Jun 17 16:17:06 localhost.localdomain manageiq[2469]:  INFO -- api:   description             : REST API
Jun 17 16:17:06 localhost.localdomain manageiq[2469]:  INFO -- api:
Jun 17 16:17:06 localhost.localdomain manageiq[2469]:  INFO -- api: Dynamic Configuration
Jun 17 16:17:06 localhost.localdomain manageiq[2469]:  INFO -- api:   token_ttl               : 10.minutes
Jun 17 16:17:06 localhost.localdomain manageiq[2469]:  INFO -- api:   authentication_timeout  : 30.seconds
Jun 17 16:17:06 localhost.localdomain manageiq[2469]:  INFO -- api:   max_results_per_page    : 1000
Jun 17 16:17:06 localhost.localdomain manageiq[2469]:  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 2469, ENV['EVMSERVER']:
Jun 17 16:19:03 localhost.localdomain systemd[1]: generic@19cf6900-4f24-4654-8f46-aad09cf51665.service: start operation timed out. Terminating.
Jun 17 16:19:03 localhost.localdomain systemd[1]: generic@19cf6900-4f24-4654-8f46-aad09cf51665.service: Failed with result 'timeout'.
Jun 17 16:19:03 localhost.localdomain systemd[1]: Failed to start generic@19cf6900-4f24-4654-8f46-aad09cf51665.service.
-- Reboot --
Jun 19 10:37:33 localhost.localdomain systemd[1]: Starting generic@19cf6900-4f24-4654-8f46-aad09cf51665.service...
Jun 19 10:38:31 localhost.localdomain bash[4024]: ** Booting MiqGenericWorker with PID: 4024 and options: {:heartbeat=>true, :guid=>"19cf6900-4f24-4654-8f46-aad09cf51665"}...
Jun 19 10:38:49 localhost.localdomain manageiq[4024]:  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
Jun 19 10:38:49 localhost.localdomain manageiq[4024]:  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
Jun 19 10:38:50 localhost.localdomain manageiq[4024]:  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- api: Initializing Environment for API
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- api:
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- api: Static Configuration
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- api:   module                  : api
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- api:   name                    : API
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- api:   description             : REST API
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- api:
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- api: Dynamic Configuration
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- api:   token_ttl               : 10.minutes
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- api:   authentication_timeout  : 30.seconds
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- api:   max_results_per_page    : 1000
Jun 19 10:38:53 localhost.localdomain manageiq[4024]:  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 4024, ENV['EVMSERVER']:
Jun 19 10:39:03 localhost.localdomain systemd[1]: generic@19cf6900-4f24-4654-8f46-aad09cf51665.service: start operation timed out. Terminating.
Jun 19 10:39:03 localhost.localdomain systemd[1]: generic@19cf6900-4f24-4654-8f46-aad09cf51665.service: Failed with result 'timeout'.
Jun 19 10:39:03 localhost.localdomain systemd[1]: Failed to start generic@19cf6900-4f24-4654-8f46-aad09cf51665.service.
Jun 19 10:40:28 localhost.localdomain systemd[1]: Starting generic@19cf6900-4f24-4654-8f46-aad09cf51665.service...
Jun 19 10:41:33 localhost.localdomain bash[4146]: ** Booting MiqGenericWorker with PID: 4146 and options: {:heartbeat=>true, :guid=>"19cf6900-4f24-4654-8f46-aad09cf51665"}...
Jun 19 10:41:52 localhost.localdomain manageiq[4146]:  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for azure.log has been changed to [WARN]
Jun 19 10:41:52 localhost.localdomain manageiq[4146]:  INFO -- evm: MIQ(Vmdb::Loggers.apply_config) Log level for vim.log has been changed to [WARN]
Jun 19 10:41:52 localhost.localdomain manageiq[4146]:  INFO -- evm: MIQ(ManageIQ::Session.configure_session_store) Using session_store: ActionDispatch::Session::MemCacheStore
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api: Initializing Environment for API
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api:
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api: Static Configuration
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api:   module                  : api
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api:   name                    : API
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api:   description             : REST API
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api:
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api: Dynamic Configuration
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api:   token_ttl               : 10.minutes
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api:   authentication_timeout  : 30.seconds
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- api:   max_results_per_page    : 1000
Jun 19 10:41:58 localhost.localdomain manageiq[4146]:  INFO -- evm: MIQ(Vmdb::Initializer.init) - Program Name: lib/workers/bin/run_single_worker.rb, PID: 4146, ENV['EVMSERVER']:
Jun 19 10:42:31 localhost.localdomain bash[4146]: /opt/manageiq/manageiq-gemset/gems/activerecord-6.0.4.4/lib/active_record/core.rb:211:in `find_by!': Couldn't find MiqGenericWorker (ActiveRecord::RecordNotFound)
Jun 19 10:42:31 localhost.localdomain bash[4146]:         from lib/workers/bin/run_single_worker.rb:115:in `<main>'
Jun 19 10:42:31 localhost.localdomain systemd[1]: generic@19cf6900-4f24-4654-8f46-aad09cf51665.service: Main process exited, code=exited, status=1/FAILURE
agrare commented 2 years ago

15GB is enough, if startup is timing out I wonder if it isn't able to hit the database. Can you try running ./tools/db_ping.rb from the /var/www/miq/vmdb directory?

miq-bot commented 1 year ago

This issue has been automatically marked as stale because it has not been updated for at least 3 months.

If you can still reproduce this issue on the current release or on master, please reply with all of the information you have about it in order to keep the issue open.

Thank you for all your contributions! More information about the ManageIQ triage process can be found in the triage process documentation.

miq-bot commented 1 year ago

This issue has been automatically closed because it has not been updated for at least 3 months.

Feel free to reopen this issue if this issue is still valid.

Thank you for all your contributions! More information about the ManageIQ triage process can be found in the triage process documentation.