Inventory Sync Job With RDS Instances Fail

[mencarellic]

ISSUE TYPE

• Bug Report

COMPONENT NAME

• UI

SUMMARY

When syncing an inventory sourced from Amazon with RDS instances, the job fails with a duplicate key error. This only seems to occur for RDS instances when no RDS instances are present in the inventory. EC2 and Elasticache instances work appropriately.

ENVIRONMENT

• AWX version: 3.0.1
• AWX install method: docker on linux
• Ansible version: 2.7.7
• Operating System: Ubuntu Linux 18.04
• Web Browser: Chrome 73.0.3683.86, FireFox 66.0

STEPS TO REPRODUCE

1. Using a Amazon EC2 Source, specify only RDS instances for the inventory using variables against ec2.ini. Make sure the overwrite and overwrite variables options are selected

   instance_states: 
   rds: true
   all_rds_instances: true

2. 1st time inventory sync works correctly.
3. Any additional inventory syncs result in failure.

EXPECTED RESULTS

Amazon RDS instances to be sync with inventory.

ACTUAL RESULTS

Python error about a duplicate key error

ADDITIONAL INFORMATION

Python Stack Trace:

  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
psycopg2.IntegrityError: duplicate key value violates unique constraint "main_host_name_inventory_id_45aecd68_uniq"
DETAIL:  Key (name, inventory_id)=(aurora_cluster_postgresql_instance_0_cplhqdemfind_us_east_1_rds_amazonaws_com, 2) already exists.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/bin/awx-manage", line 11, in <module>
    load_entry_point('awx==3.0.1.0', 'console_scripts', 'awx-manage')()
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/__init__.py", line 140, in manage
    execute_from_command_line(sys.argv)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 1049, in handle
    raise exc
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 994, in handle
    self.load_into_database()
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 848, in load_into_database
    self._create_update_hosts()
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 727, in _create_update_hosts
    self._update_db_host_from_mem_host(db_host, mem_host)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 663, in _update_db_host_from_mem_host
    db_host.save(update_fields=update_fields)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/inventory.py", line 732, in save
    super(Host, self).save(*args, **kwargs)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/base.py", line 316, in save
    super(PrimordialModel, self).save(*args, **kwargs)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/base.py", line 164, in save
    super(CreatedModifiedModel, self).save(*args, **kwargs)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/base.py", line 808, in save
    force_update=force_update, update_fields=update_fields)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/base.py", line 838, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/base.py", line 905, in _save_table
    forced_update)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/base.py", line 955, in _do_update
    return filtered._update(values) > 0
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/query.py", line 667, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 1204, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 899, in execute_sql
    raise original_exception
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/models/sql/compiler.py", line 889, in execute_sql
    cursor.execute(sql, params)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "main_host_name_inventory_id_45aecd68_uniq"
DETAIL:  Key (name, inventory_id)=(aurora_cluster_postgresql_instance_0_cplhqdemfind_us_east_1_rds_amazonaws_com, 2) already exists.

[sugitk]

inventory_update_247.txt

I also reproduced like this issue on VMware ESXi. At first the inventory sync was successful, but after 2nd try always fails due to duplicate key.

Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
psycopg2.IntegrityError: duplicate key value violates unique constraint "main_host_name_inventory_id_45aecd68_uniq"
DETAIL:  Key (name, inventory_id)=(RHEL7.6_Tower_DB_564d838d-f777-9a35-28e3-536b8151922a, 4) already exists.

[shanemcd]

Closing due to lack of activity. If this is still a problem please open a new issue or reach out on the mailing list.