search

Callum027 / ipam-migrator

Data migration tool for between phpIPAM and NetBox
GNU General Public License v3.0
17 stars 8 forks source link

KeyError: 34 #8

Open joubbi opened 2 years ago

joubbi commented 2 years ago

I used the fork https://github.com/absynth47215/ipam-migrator Thank you @Callum027 and @absynth47215 for sharing your work!

I have phpIPAM v1.4.5 and NetBox v3.1.5

When I do a migration with the following command:

$ ipam-migrator --log-level DEBUG -naisv -noasv https://localhost/api/netbox,phpipam,login,username,ipamadmin https://192.168.0.2/api,netbox,key,secretkey"

It connects fine to both servers but fails after a while when writing data to NetBox with the following message:

2022-01-19 14:45:58,484 ipam-migrator: [INFO] Wrote 785 VLANs.

2022-01-19 14:45:58,487 ipam-migrator: [INFO] Writing prefixes...

2022-01-19 14:45:58,734 ipam-migrator: [DEBUG] updated prefix 10.x.x.x/29 with description 'albatross_server'

2022-01-19 14:45:58,972 ipam-migrator: [DEBUG] updated prefix 10.x.x.x/28 with description 'albatross_server2'

2022-01-19 14:45:58,973 ipam-migrator: [ERROR] 34

Traceback (most recent call last):

  File "/usr/local/lib/python3.9/dist-packages/ipam_migrator-1.0.0-py3.9.egg/ipam_migrator/ipam_migrator.py", line 183, in main

    output_backend.database_write(input_database)

  File "/usr/local/lib/python3.9/dist-packages/ipam_migrator-1.0.0-py3.9.egg/ipam_migrator/backend/netbox.py", line 388, in database_write

    prefixes_new, prefixes_old_to_new = self.prefixes_write(

  File "/usr/local/lib/python3.9/dist-packages/ipam_migrator-1.0.0-py3.9.egg/ipam_migrator/backend/netbox.py", line 511, in prefixes_write

    "vrf": vrfs_old_to_new[prefix.vrf_id] if prefix.vrf_id else None,

KeyError: 34

2022-01-19 14:45:58,998 ipam-migrator: [DEBUG] stopping logger

Let me know if there is something I can do to help debugging this.

Absynth723 commented 2 years ago

I'm not sure what happens there. I think that the script is trying to access an array key that simply doesn't exist. You should be able to simply comment out line 511, which is "vrf": vrfs_old_to_new[prefix.vrf_id] if prefix.vrf_id else None, That would result in no vrf ID being written to your ip address in Netbox, but since the VRF ID is not mandatory for IP addresses in Netbox's IPAM module, the migration should still work. I didn't see this error exporting my stuff from phpIPAM to Netbox so I'm not sure why you are.

joubbi commented 2 years ago

I commented out line number 511 in src/ipam_migrator/backend/netbox.py. The migration goes through without any errors.

No VRFs were migrated. This is probably as expected since I commented out the line? But the imported prefixes do not have a VLAN either. Is this a shortcoming of the importer or is there something wrong?

Absynth723 commented 2 years ago

yes, VRFs aren't implemented, as far as I can tell. The VRF-related methods are programmed to give a "Not implemented" type of error. However, my prefixes have VLANs associated with them so I'm not entirely sure why yours don't. Did you try re-running the script? Maybe it's a timing issue and once all VLANs are migrated, the prefixes get migrated on the second attempt or so.

joubbi commented 2 years ago

If line number 511 doesn't make any difference, maybe you should remove it from your repo?

I ran the migration one more time. I can see in the Change Log of netbox for a prefix that it was updated. I have a subnet in phpipam 10.x.x.x.x/31 with VLAN 3143. After the migration I have 10.x.x.x/31 as a Prefix in netbox. That Prefix doesn't have a VLAN. After the migration I have VLAN 3143 in netbox. Looking at the Change Log in netbox for that VLAN, I can see that it gets updated twice every time I run ipam-migrator.

Have you tested that your prefixes have a VLAN associated after fixing the VLAN bug yesterday?

Absynth723 commented 2 years ago

I haven't fixed any bug yesterday, I have just committed my most recent local changes back upstream. I'm currently too busy to test any further, so I'd recommend you fork the repo and try fixing it up to your requirements yourself.

joubbi commented 2 years ago

Thank you for your help so far.

Which versions of netbox and phpipam were you using when you did the import? Maybe something changed again.

Absynth723 commented 2 years ago

My test installations are Netbox 3.0.11 (latest would be 3.1.6 or something) and phpIPAM 1.5.0.

joubbi commented 2 years ago

I upgraded my phpIPAM to 1.5.0 r38. That made the ipam-migrator do it's thing a little faster since it didn't pause the same way it did earlier. But the end result was the same.

I then installed netbox-docker 1.4.1, which runs netbox v3.0.11. The result was the exact same after running ipam-migrator. I needed to comment line 511 and the prefixes don't have a VLAN.