cbka
commented
9 months ago +1
Open howardtopher opened 9 months ago
cbka
commented
9 months ago +1
howardtopher
commented
9 months ago I did determine that the import works if you specify a vendor to install. I only encounter the error when running the script to install everything at once.
Konraden
commented
8 months ago The device library itself has "Watchguard" and "WatchGuard" directories containing devices. I'm going guess it breaks when it tires to create a slug the second time. The index is likely case insensitive but the script isn't.
gurubert
commented
8 months ago https://github.com/netbox-community/devicetype-library/pull/1978 fixes this issue.
netravnen
commented
8 months ago Used the master branch of the repository here for the importer. Tried limiting the vendors to different ones. Without any real success.
# /opt/Device-Type-Library-Import/venv/bin/python3 /opt/Device-Type-Library-Import/nb-dt-import.py --vendors Generic
Package devicetype-library is already installed, updating /opt/Device-Type-Library-Import/repo
Traceback (most recent call last):
File "/opt/Device-Type-Library-Import/nb-dt-import.py", line 54, in <module>
main()
File "/opt/Device-Type-Library-Import/nb-dt-import.py", line 17, in main
netbox = NetBox(settings)
^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/netbox_api.py", line 29, in __init__
self.existing_manufacturers = self.get_manufacturers()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in get_manufacturers
return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in <dictcomp>
return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/response.py", line 127, in __next__
next(self.response), self.endpoint.api, self.endpoint
^^^^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 280, in get
req = self._make_call(add_params=add_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 247, in _make_call
raise RequestError(req)
pynetbox.core.query.RequestError: The request failed with code 500 Internal Server Error but more specific details were not returned in json. Check the NetBox Logs or investigate this exception's error attribute.# ./nb-dt-import.py --vendors apc
Package devicetype-library is already installed, updating /opt/Device-Type-Library-Import/repo
Traceback (most recent call last):
File "/opt/Device-Type-Library-Import/./nb-dt-import.py", line 54, in <module>
main()
File "/opt/Device-Type-Library-Import/./nb-dt-import.py", line 17, in main
netbox = NetBox(settings)
^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/netbox_api.py", line 29, in __init__
self.existing_manufacturers = self.get_manufacturers()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in get_manufacturers
return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in <dictcomp>
return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/response.py", line 127, in __next__
next(self.response), self.endpoint.api, self.endpoint
^^^^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 280, in get
req = self._make_call(add_params=add_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 247, in _make_call
raise RequestError(req)
pynetbox.core.query.RequestError: The request failed with code 500 Internal Server Error but more specific details were not returned in json. Check the NetBox Logs or investigate this exception's error attribute.
danner26
commented
8 months ago v3 is supported, but a newer version must have broke this functionality. We will look into it ASAP, or anyone is welcome to submit a PR if the issue is found
gurubert
commented
8 months ago The PR is already merged: https://github.com/netbox-community/devicetype-library/pull/1978
netravnen
commented
8 months ago v3 is supported, but a newer version must have broke this functionality. We will look into it ASAP, or anyone is welcome to submit a PR if the issue is found
Now double-checking, after your response, and ensuring I have pulled the latest import and repo commits. It works... To a degree.
Now I am hitting different kind of errors during the device type import.
conversion between UTF8 and SQL_ASCII is not supported ⟵ Unsure how to got about this one. Installed the netbox in an LXC container according to the manual. Debian 15.6-0+deb12u1.
SQL_ASCII if not specified otherwise.template1 to new default encoding (UTF8).power_port":["Related objects must be referenced by numeric ID or by dictionary of attributes. Received an unrecognized value: PS1" ⟵ Error {"u_height":["Child device types must be 0U."]} creating device type: Mellanox SX6012T-2BFS ⟵ Add u_height value to yaml definition. https://github.com/netbox-community/devicetype-library/pull/2014Could not find Rear Port for Front Port: Port/18 - st - 1594 Error '[{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{"rear_port":["Related objects must be referenced by numeric ID or by dictionary of attributes. Received an unrecognized value: Port/18"]},{},{},{},{},{},{}]' creating Front Port ⟵ ... 🤔 💭 https://github.com/netbox-community/devicetype-library/pull/2015
mhdan
commented
4 weeks ago I did determine that the import works if you specify a vendor to install. I only encounter the error when running the script to install everything at once.
having the same problem, the https://github.com/netbox-community/devicetype-library/pull/1978 PR don't fix this issue and I encounter the issue when trying to import all devices but it's working when trying each vendor standalone.
here is verbose output of error in my case
(venv) root@netbox:/opt/Device-Type-Library-Import# ./nb-dt-import.py --verbose [608/1806]Package devicetype-library is already installed, updating /opt/Device-Type-Library-Import/repo
Pulled Repo https://github.com/netbox-community/devicetype-library.git Netbox version 4.1 found. Using new filters.
143 Vendors Found 3534 Device-Types Found
Manufacturer Exists: AFL - 11
Manufacturer queued for addition: Check Point
Manufacturer queued for addition: Rockwell Automation Manufacturer queued for addition: Supermicro
Manufacturer queued for addition: QNAP Manufacturer queued for addition: Nvidia
Manufacturer queued for addition: Panduit Manufacturer queued for addition: D-Link
Manufacturer queued for addition: CryptoPro Manufacturer queued for addition: Delta
Manufacturer queued for addition: Moxa Manufacturer queued for addition: ZTE
Manufacturer Exists: Arista - 2 Manufacturer queued for addition: ActionTec
Manufacturer queued for addition: NetApp
Manufacturer queued for addition: TrendNet Manufacturer queued for addition: Alpha
Manufacturer queued for addition: Commscope Manufacturer queued for addition: Schneider Electric
Manufacturer queued for addition: Diamond SA Manufacturer queued for addition: Fujitsu
Manufacturer queued for addition: Palo Alto Manufacturer queued for addition: Avocent
Manufacturer queued for addition: Peplink Manufacturer queued for addition: Canon
Manufacturer queued for addition: Factor-TS Manufacturer queued for addition: LANCOM
Manufacturer queued for addition: Datto
Manufacturer queued for addition: Digi
Manufacturer queued for addition: Colable
Manufacturer queued for addition: WatchGuard
Manufacturer queued for addition: Opengear
Manufacturer queued for addition: Adtran
Manufacturer queued for addition: Qotom
Manufacturer queued for addition: Digitus
Manufacturer queued for addition: Infoblox
Manufacturer queued for addition: QCT
Manufacturer queued for addition: Citrix
Manufacturer queued for addition: CE6885-48YS8CQ.yaml
Manufacturer queued for addition: Sophos
Manufacturer queued for addition: WTI
Manufacturer queued for addition: Calient
Manufacturer queued for addition: Inspur
Manufacturer queued for addition: QTECH
Manufacturer queued for addition: Server Technology
Manufacturer queued for addition: Pluribus
Manufacturer queued for addition: Nozomi Networks
Manufacturer queued for addition: Arctic-Wolf
Manufacturer queued for addition: Brocade
Manufacturer queued for addition: Infinera
Manufacturer queued for addition: Netgear
Manufacturer queued for addition: Ippon
Manufacturer queued for addition: F5
Manufacturer queued for addition: Netonix
Manufacturer queued for addition: Ciena
Manufacturer Exists: Juniper - 1
Manufacturer queued for addition: Eaton
Manufacturer Exists: Cisco - 8
Manufacturer queued for addition: Ericsson
Manufacturer queued for addition: Apple
Manufacturer queued for addition: Thales
Manufacturer queued for addition: Riverbed
Manufacturer queued for addition: Ruckus
Manufacturer queued for addition: Solid Optics
Manufacturer queued for addition: Deciso
Manufacturer queued for addition: ADVA
Manufacturer queued for addition: Datwyler
Manufacturer queued for addition: Mimosa Networks
Manufacturer queued for addition: Intel
Manufacturer queued for addition: Huawei
Manufacturer queued for addition: Hikvision
Manufacturer queued for addition: Leviton
Manufacturer Exists: Dell - 9
Manufacturer queued for addition: ZPE
Manufacturer queued for addition: SonicWall
Manufacturer queued for addition: Corning
Manufacturer queued for addition: CyberPower
Manufacturer queued for addition: Fortinet
Manufacturer queued for addition: Riello
Manufacturer queued for addition: Zonit
Manufacturer queued for addition: Rohde & Schwarz
Manufacturer queued for addition: nexconec
Manufacturer queued for addition: Datacom
Manufacturer queued for addition: A10
Manufacturer queued for addition: Middle Atlantic
Manufacturer queued for addition: Infotecs
Manufacturer queued for addition: DataDirect Networks
Manufacturer queued for addition: Netgate
Manufacturer queued for addition: Generic
Manufacturer queued for addition: TP-Link
Manufacturer queued for addition: LiteOn
Manufacturer queued for addition: MikroTik
Manufacturer queued for addition: Lenovo
Manufacturer queued for addition: Cuda
Manufacturer queued for addition: RAD
Manufacturer queued for addition: APC
Manufacturer queued for addition: Parks
Manufacturer queued for addition: Mellanox
Manufacturer queued for addition: EnGenius
Manufacturer queued for addition: FS
Manufacturer queued for addition: Albis-Elcon
Manufacturer queued for addition: IBM
Manufacturer queued for addition: Nokia
Manufacturer queued for addition: Zyxel
Manufacturer queued for addition: ghipsystems
Manufacturer queued for addition: Overland-Tandberg
Manufacturer queued for addition: Synology
Manufacturer queued for addition: Eltex
Manufacturer queued for addition: Blackmagicdesign
Manufacturer queued for addition: Barracuda
Manufacturer queued for addition: Raspberry Pi
Manufacturer Exists: HPE - 10
Manufacturer queued for addition: PC Engines
Manufacturer queued for addition: Alcatel-Lucent
Manufacturer queued for addition: Reichle & De-Massari
Manufacturer queued for addition: Aten
Manufacturer queued for addition: Bachmann
Manufacturer queued for addition: YAMAHA
Manufacturer queued for addition: SNR
Manufacturer queued for addition: Rittal
Manufacturer queued for addition: Sedna
Manufacturer queued for addition: DCN
Manufacturer queued for addition: Edgecore
Manufacturer queued for addition: Allied Telesis
Manufacturer queued for addition: Neousys
Manufacturer queued for addition: Teltonika
Manufacturer queued for addition: Digital Loggers
Manufacturer queued for addition: ExaGrid
Manufacturer queued for addition: Brennenstuhl
Manufacturer queued for addition: AVM
Manufacturer queued for addition: Calix
Manufacturer queued for addition: NEC
Manufacturer queued for addition: 3Com
Manufacturer queued for addition: Ubiquiti
Manufacturer queued for addition: V-Solution
Manufacturer queued for addition: Cradlepoint
Manufacturer queued for addition: Ivanti
Manufacturer queued for addition: Extreme Networks
Manufacturer queued for addition: T-bao
Manufacturer queued for addition: Certa
Manufacturer queued for addition: FireBrick
Manufacturer queued for addition: Raritan
Manufacturer queued for addition: Vertiv
Error creating manufacturers
Error during manufacturer creation. - [{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{}
,{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{"slug":["manufacturer with this slug already exists."]},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{
},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{}]
Device Type Exists: AFL - U-Series Housing 2RU, 12 Cassettes - 1652
Device Type Exists: AFL - U-Series Housing 1RU, 5 Cassettes - 1653
Error {"manufacturer":["Related object not found using the provided attributes: {'name': 'Check Point', 'slug': 'check-point'}"]} creating device type: Check Point SG6400
Error {"manufacturer":["Related object not found using the provided attributes: {'name': 'Check Point', 'slug': 'check-point'}"]} creating device type: Check Point SG23800
Error {"manufacturer":["Related object not found using the provided attributes: {'name': 'Check Point', 'slug': 'check-point'}"]} creating device type: Check Point SG3600
Error {"manufacturer":["Related object not found using the provided attributes: {'name': 'Check Point', 'slug': 'check-point'}"]} creating device type: Check Point CPAC-1500/3600/3800-RM-DUAL
My guess here is that this script doesn't support netbox v3. But this is the error I get when trying to run the script to do an initial import on a new system. There are currently 0 manufacturers existing on this system yet.
I tried changing requirements.txt to bump pynetbox up to 7.3.3 and I still get the same error.