peeringdb sync fails with foreign key constraint error

[ddrennan-dch]

Hi, I'm trying to run peeringdb sync on Windows 10 into a mysql db, and even after a fresh drop-tables it is failing to load some record in the facility table:

Syncing to https://www.peeringdb.com/api Updating resources: org fac net ix ixfac ixlan ixpfx netfac netixlan poc Fetching & updating all: org Updates to be processed: 19332 Ignoring object updated after sync began: (org-852) Ignoring object updated after sync began: (org-11809) Ignoring object updated after sync began: (org-17077) Ignoring object updated after sync began: (org-27027) Ignoring object updated after sync began: (org-27046) Ignoring object updated after sync began: (org-27047) Ignoring object updated after sync began: (org-27048) Ignoring object updated after sync began: (org-27049) Ignoring object updated after sync began: (org-27050) Fetching & updating all: fac Updates to be processed: 3836 Traceback (most recent call last): File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\backends\mysql\base.py", line 71, in execute return self.cursor.execute(query, args) File "c:\users\ddrennan\pdbenv\lib\site-packages\MySQLdb\cursors.py", line 206, in execute res = self._query(query) File "c:\users\ddrennan\pdbenv\lib\site-packages\MySQLdb\cursors.py", line 319, in _query db.query(q) File "c:\users\ddrennan\pdbenv\lib\site-packages\MySQLdb\connections.py", line 259, in query _mysql.connection.query(self, query) MySQLdb._exceptions.IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (peeringdb.peeringdb_facility, CONSTRAINT peeringdb_f_org_id_35cdb86b2209de15_fk_peeringdb_organization_id FOREIGN KEY (org_id) REFERENCES peeringdb_organization (id))')

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "c:\program files\python36\Lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "c:\program files\python36\Lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\ddrennan\pdbenv\Scripts\peeringdb.exe__main.py", line 9, in File "c:\users\ddrennan\pdbenv\lib\site-packages\peeringdb\cli.py", line 62, in main return handler(config=cfg, vars(options)) File "c:\users\ddrennan\pdbenv\lib\site-packages\peeringdb\commands.py", line 20, in _wrapped r = func(*a, *k) File "c:\users\ddrennan\pdbenv\lib\site-packages\peeringdb\commands.py", line 219, in handle client.update_all(rs) File "c:\users\ddrennan\pdbenv\lib\site-packages\peeringdb_update.py", line 66, in update_all self._atomic_update(lambda: ctx.sync_resource(r, since=since)) File "c:\users\ddrennan\pdbenv\lib\site-packages\peeringdb_update.py", line 78, in _atomic_update sync_func() File "c:\users\ddrennan\pdbenv\lib\site-packages\peeringdb_update.py", line 66, in self._atomic_update(lambda: ctx.sync_resource(r, since=since)) File "c:\users\ddrennan\pdbenv\lib\site-packages\peeringdb_tasks_async.py", line 65, in _wrapped return loop.run_until_complete(func(a, k)) File "c:\program files\python36\Lib\asyncio\base_events.py", line 467, in run_until_complete return future.result() File "c:\users\ddrennan\pdbenv\lib\site-packages\peeringdb_tasks_async.py", line 41, in _wrapped item = gen.send(r) File "c:\users\ddrennan\pdbenv\lib\site-packages\peeringdb_update.py", line 274, in sync_row B.save(obj) File "c:\users\ddrennan\pdbenv\lib\site-packages\django_peeringdb\client_adaptor\backend.py", line 153, in save obj.save() File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\models\base.py", line 741, in save force_update=force_update, update_fields=update_fields) File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\models\base.py", line 779, in save_base force_update, using, update_fields, File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\models\base.py", line 870, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\models\base.py", line 908, in _do_insert using=using, raw=raw) File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\models\manager.py", line 82, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\models\query.py", line 1186, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\models\sql\compiler.py", line 1375, in execute_sql cursor.execute(sql, params) File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\backends\utils.py", line 67, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\backends\utils.py", line 76, in _execute_with_wrappers return executor(sql, params, many, context) File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\utils.py", line 89, in exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "c:\users\ddrennan\pdbenv\lib\site-packages\django\db\backends\mysql\base.py", line 71, in execute return self.cursor.execute(query, args) File "c:\users\ddrennan\pdbenv\lib\site-packages\MySQLdb\cursors.py", line 206, in execute res = self._query(query) File "c:\users\ddrennan\pdbenv\lib\site-packages\MySQLdb\cursors.py", line 319, in _query db.query(q) File "c:\users\ddrennan\pdbenv\lib\site-packages\MySQLdb\connections.py", line 259, in query _mysql.connection.query(self, query) django.db.utils.IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (peeringdb.peeringdb_facility, CONSTRAINT peeringdb_f_org_id_35cdb86b2209de15_fk_peeringdb_organization_id FOREIGN KEY (org_id) REFERENCES peeringdb_organization (id))')

Any suggestions?

[vegu]

I believe this is the same as #41 - the recent release of peeringdb-py 1.1.0 and django-peeringdb 2.3.0 should address this, can you try installing those and let us know if that fixes the issue for you? thanks

[grizz]

Assuming this is fixed, please re-open if you are still getting this error.