peeringdb / peeringdb-py

PeeringDB python client
BSD 2-Clause "Simplified" License
89 stars 22 forks source link

Error when synchronizing peeringdb #1

Closed Tigerdyr closed 8 years ago

Tigerdyr commented 8 years ago

I've setup virtual python Environment. The installation was quite painless but the sync fails, I don't know if that is due to error in the source database or what, but it fails. Everything runs smoothly until I get to the netixlan, here is seems there is an ix lan missing - ID 585. Reported this to support, was asked to create an issue :), So here it is.

command peeringdb sync Log of what is happening: ... cut to failing part ... Starting new HTTPS connection (1): beta.peeringdb.com "GET /api/netixlan?since=0 HTTP/1.1" 200 None netixlan last update 0 15931 changed data to be processed 15931 Traceback (most recent call last): File "/home/sps/peeringdb/venv/bin/peeringdb", line 11, in sys.exit(cli()) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/click/core.py", line 716, in call return self.main(_args, _kwargs) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/click/core.py", line 696, in main rv = self.invoke(ctx) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/click/core.py", line 1060, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/click/core.py", line 889, in invoke return ctx.invoke(self.callback, _ctx.params) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke return callback(_args, *kwargs) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/peeringdb/cli.py", line 125, in sync db.sync() File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/peeringdb/localdb.py", line 97, in sync call_command('pdb_sync', interactive=False) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/django/core/management/init.py", line 120, in call_command return command.execute(_args, _defaults) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 441, in execute output = self.handle(_args, *_options) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/django_peeringdb/management/commands/pdb_sync.py", line 85, in handle self.sync(tables) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/django_peeringdb/management/commands/pdb_sync.py", line 92, in sync self.update_db(cls, self.get_objs(cls)) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/django_peeringdb/management/commands/pdb_sync.py", line 127, in update_db sync.sync_obj(cls, row) File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/django_peeringdb/sync.py", line 25, in sync_obj obj.full_clean() File "/home/sps/peeringdb/venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 1147, in full_clean raise ValidationError(errors) django.core.exceptions.ValidationError: {'ixlan': [u'ix lan instance with id 585 does not exist.']}

grizz commented 8 years ago

What version of Django are you using?

pip freeze | grep Django
Tigerdyr commented 8 years ago

Ah, my bad - should've had this with in first place :)

(venv) sps@tools:~/peeringdb$ pip freeze click==6.2 Django==1.8.1 django-countries==3.4.1 django-handleref==0.1.4 django-inet==0.1.3 django-peeringdb==0.2.0 ipaddress==1.0.16 munge==0.1.7 peeringdb==0.4.2 PyYAML==3.11 requests==2.9.1 twentyc.rpc==0.3.4 wheel==0.26.0

(venv) sps@tools:~/.peeringdb$ cat ~/.peeringdb/config.yaml database: engine: sqlite3 name: peeringdb.sqlite3 peeringdb: password: '' timeout: 0 url: https://beta.peeringdb.com/api user: ''

Let me know if you need anything more

vegu commented 8 years ago

This issue appears to have been caused by bad data and should be fixed for the time being