search

dainok / netdoc

Automatic Network Documentation plugin for NetBox
GNU General Public License v3.0
91 stars 14 forks source link

Discovery of VMware vSphere device has error vim.dvs.PortConnection is not JSON serializable #75

Closed dietybright closed 1 year ago

dietybright commented 1 year ago

When running discovery vcenter/esxi-host with admin credential, I am facing issue with netdoc 3.5.3 or 0.10.33, any idea?

An exception occurred: TypeError: Object of type vim.dvs.PortConnection is not JSON serializable

Traceback (most recent call last): File "/opt/netbox/netbox/extras/scripts.py", line 504, in _run_script script.output = script.run(data=data, commit=commit) File "/opt/netbox/netbox/scripts/netdoc_scripts.py", line 262, in run output = discovery( File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/tasks.py", line 111, in discovery module.discovery(filtered_devices, filters=filters, filter_type=filter_type) File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/discoverers/json_vmware_vsphere.py", line 272, in discovery raw_output=json.dumps(result.result), File "/usr/lib/python3.10/json/init.py", line 231, in dumps return _default_encoder.encode(obj) File "/usr/lib/python3.10/json/encoder.py", line 199, in encode chunks = self.iterencode(o, _one_shot=True) File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode return _iterencode(o, 0) File "/usr/lib/python3.10/json/encoder.py", line 179, in default raise TypeError(f'Object of type {o.class.name} ' TypeError: Object of type vim.dvs.PortConnection is not JSON serializable

dainok commented 1 year ago

Unfortunately I don't have a DVS on my lab so I cannot debug this issue. Can you debug /opt/netbox/venv/lib/python3.10/site-packages/netdoc/discoverers/json_vmware_vsphere.py ?

dietybright commented 1 year ago

This is the debug output, not sure if it helpful

INFO:nornir.core:Running task 'multiple_tasks' with args {} on 1 hosts DEBUG:nornir.core.task:Host '192.168.1.53': running task 'multiple_tasks' DEBUG:nornir.core.task:Host '192.168.1.53': running task '{"command": "pyVmomi", "template": "pyVmomi", "order": 0, "enable": false, "supported": true, "verify_cert": false}' ERROR:netbox.scripts.NetDoc.Discover:An exception occurred: TypeError: Object of type vim.dvs.PortConnection is not JSON serializable

Traceback (most recent call last):
  File "/opt/netbox/netbox/extras/scripts.py", line 476, in _run_script
    script.output = script.run(data=data, commit=commit)
  File "/opt/netbox/netbox/scripts/NetDoc.py", line 264, in run
    output = discovery(
  File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/tasks.py", line 111, in discovery
    module.discovery(filtered_devices, filters=filters, filter_type=filter_type)
  File "/opt/netbox/venv/lib/python3.10/site-packages/netdoc/discoverers/json_vmware_vsphere.py", line 280, in discovery
    raw_output=json.dumps(result.result),
  File "/usr/lib/python3.10/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type vim.dvs.PortConnection is not JSON serializable

INFO:netbox.scripts.NetDoc.Discover:Database changes have been reverted due to error. ERROR:netbox.scripts.NetDoc.Discover:Exception raised during script execution: Object of type vim.dvs.PortConnection is not JSON serializable INFO:webhooks:Clearing 0 queued webhooks (<utilities.utils.NetBoxFakeRequest object at 0x7f8e0614b490>) INFO:netbox.scripts.NetDoc.Discover:Script completed in 0 minutes, 13.96 seconds DEBUG:netbox.config:No previous configuration found in database; proceeding with default values DEBUG:netbox.config:Initialized configuration INFO:rq.worker:default: Job OK (a727df2b-962c-4108-b92a-3fd1ccf228c0) INFO:rq.worker:Result is kept for 500 seconds

cmong-nextech commented 1 year ago

Try replacing line 272 in "/opt/netbox/venv/lib/python3.9/site-packages/netdoc/discoverers/json_vmware_vsphere.py" raw_output=json.dumps(result.result), with raw_output=json.dumps(result.result, default=lambda o: o.__dict__), That got it working for me.

dainok commented 1 year ago

Thank you. Let me fix a major issue then I'll merge your suggestion.

dainok commented 1 year ago

Merged with https://github.com/dainok/netdoc/pull/80 Give a while for the CI/CD, upgrade to 3.5.6 and let me know.

dietybright commented 1 year ago

Its worked with 3.5.6 but my enviroment unable to get vswitch/portgroup, seem we need to define on netbox first

Another thing is the connected interface/device of host physical adapter via pyVmomi cdp neighbor information, it would be greate if netdoc can make this connection

dainok commented 1 year ago

Currently I'm using the NetBox data structure to store clusters, hosts and VMs. I'm not completely satisfied because all network information are lost. NetBox does not have the vSwitch concept.

Regarding host CDP, if you can modify the discover script, to grab those info, I can modify the ingest script.

dainok commented 1 year ago

In case, open a new issue to track improvements.