search

peeringdb / peeringdb-py

PeeringDB python client
BSD 2-Clause "Simplified" License
89 stars 22 forks source link

Handle schema divergence errors #59

Closed vegu closed 3 months ago

vegu commented 3 years ago

If one runs a local snapshot of the django-peeringdb schema and then makes schema additions (e.g., adding a foreign key field) that has the potential to cause sync issues - it seems to be peeringdb-py is processing those new fields like they should get data synced to them when they shouldn't.

  File "/srv/www.peeringdb.com/./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/srv/www.peeringdb.com/peeringdb_server/management/commands/pdb_load_data.py", line 116, in handle
    client.update_all(resource.all_resources(), since=None)
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/peeringdb/_update.py", line 68, in update_all
    ctx.sync_resource(r, since=since)
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/peeringdb/_tasks_sequential.py", line 76, in _wrapped
    return _consume_task(gen)
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/peeringdb/_tasks_sequential.py", line 57, in _consume_task
    item = gen.send(r)
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/peeringdb/_tasks_sequential.py", line 38, in gather
    yield from job
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/peeringdb/_update.py", line 199, in sync_row
    fetched, dangling = _sync.extract_relations(B, res, row)
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/peeringdb/_sync.py", line 89, in extract_relations
    _, subrow = _get_subrow(row, fname, field)
  File "/srv/www.peeringdb.com/venv/lib/python3.9/site-packages/peeringdb/_sync.py", line 68, in _get_subrow
    subrow = row[key]
KeyError: 'ixf_import_request_user'

Probably just need to handle KeyError in _get_subrow

leovegoda commented 3 months ago

Closing as this can no longer be reproduced following significant code changes