search

pryorda / vmware_exporter

VMWare vCenter Exporter for Prometheus
BSD 3-Clause "New" or "Revised" License
519 stars 199 forks source link

Exporter Failing to work with Vcentre 7.0.3 #341

Open halamalagarli opened 1 year ago

halamalagarli commented 1 year ago

Hi

Thanks you for your efforts. I am running vcentre 7.0.3 on hosts running Esxi 6.7 on machine with the exporter I am running python 3.7.3. The config file that I tried is as follows

default: vsphere_host: "vcenter1" vsphere_user: "user" vsphere_password: "password" ignore_ssl: True specs_size: 5000 fetch_custom_attributes: False fetch_tags: False fetch_alarms: False collect_only: vms: False vmguests: False datastores: False hosts: True snapshots: False

When I run vmware_exporter -c config.yml I get the following traceback

2022-11-15 15:21:44,509 INFO:Starting web server on port :9272 2022-11-15 15:21:51,381 INFO:Start collecting metrics from vcenter1.campus.bath.ac.uk 2022-11-15 15:21:51,381 INFO:Starting host metrics collection 2022-11-15 15:21:51,381 INFO:Fetching vim.HostSystem inventory 2022-11-15 15:21:51,382 INFO:Retrieving service instance content 2022-11-15 15:21:51,383 INFO:START: _vmware_get_host_perf_manager_metrics 2022-11-15 15:21:55,670 ERROR:Caught vmodl fault: Cannot complete login due to an incorrect user name or password. 2022-11-15 15:21:55,671 ERROR:Traceback (most recent call last): File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 1937, in _async_render_GET yield self.generate_latest_metrics(request) twisted.internet.defer.FirstError: FirstError[#0, [Failure instance: Traceback: <class 'twisted.internet.defer.FirstError'>: FirstError[#0, [Failure instance: Traceback: <class 'AttributeError'>: 'NoneType' object has no attribute 'RetrieveContent' /var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py:735:errback /var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py:798:_startRunCallbacks /var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py:893:_runCallbacks /var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py:1792:gotResult --- --- /var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py:1694:_inlineCallbacks /var/tmp/.venv/lib/python3.7/site-packages/twisted/python/failure.py:518:throwExceptionIntoGenerator /var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py:709:host_system_inventory /var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py:1694:_inlineCallbacks /var/tmp/.venv/lib/python3.7/site-packages/twisted/python/failure.py:518:throwExceptionIntoGenerator /var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py:602:batch_fetch_properties /var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py:1697:_inlineCallbacks /var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py:594:content ]] --- --- /var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py:1642:_vmware_get_hosts /var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/defer.py:100:parallelize ]]

Unhandled error in Deferred:

Traceback (most recent call last): File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 735, in errback self._startRunCallbacks(fail) File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks self._runCallbacks() File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 893, in _runCallbacks current.result, *args, **kwargs File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1792, in gotResult _inlineCallbacks(r, gen, status, context) --- --- File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1694, in _inlineCallbacks cast(Failure, result).throwExceptionIntoGenerator, gen File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 709, in host_system_inventory properties, File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1694, in _inlineCallbacks cast(Failure, result).throwExceptionIntoGenerator, gen File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 602, in batch_fetch_properties content = yield self.content File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks result = context.run(gen.send, result) File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 594, in content connection.RetrieveContent builtins.AttributeError: 'NoneType' object has no attribute 'RetrieveContent'

Unhandled error in Deferred:

Traceback (most recent call last): File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 735, in errback self._startRunCallbacks(fail) File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks self._runCallbacks() File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 893, in _runCallbacks current.result, *args, **kwargs File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1792, in gotResult _inlineCallbacks(r, gen, status, context) --- --- File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1694, in _inlineCallbacks cast(Failure, result).throwExceptionIntoGenerator, gen File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 709, in host_system_inventory properties, File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1694, in _inlineCallbacks cast(Failure, result).throwExceptionIntoGenerator, gen File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 602, in batch_fetch_properties content = yield self.content File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks result = context.run(gen.send, result) File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 594, in content connection.RetrieveContent builtins.AttributeError: 'NoneType' object has no attribute 'RetrieveContent'

Unhandled error in Deferred:

Traceback (most recent call last): File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 735, in errback self._startRunCallbacks(fail) File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks self._runCallbacks() File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 893, in _runCallbacks current.result, *args, **kwargs File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1792, in gotResult _inlineCallbacks(r, gen, status, context) --- --- File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1694, in _inlineCallbacks cast(Failure, result).throwExceptionIntoGenerator, gen File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 1038, in host_labels dcs = yield self.datacenter_inventory File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1694, in _inlineCallbacks cast(Failure, result).throwExceptionIntoGenerator, gen File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 970, in datacenter_inventory content = yield self.content File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks result = context.run(gen.send, result) File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 594, in content connection.RetrieveContent builtins.AttributeError: 'NoneType' object has no attribute 'RetrieveContent'

Unhandled error in Deferred:

Traceback (most recent call last): File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 735, in errback self._startRunCallbacks(fail) File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks self._runCallbacks() File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 893, in _runCallbacks current.result, *args, **kwargs File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1792, in gotResult _inlineCallbacks(r, gen, status, context) --- --- File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1694, in _inlineCallbacks cast(Failure, result).throwExceptionIntoGenerator, gen File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 1138, in counter_ids content = yield self.content File "/var/tmp/.venv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks result = context.run(gen.send, result) File "/var/tmp/.venv/lib/python3.7/site-packages/vmware_exporter/vmware_exporter.py", line 594, in content connection.RetrieveContent builtins.AttributeError: 'NoneType' object has no attribute 'RetrieveContent'

Can you please suggest where I can begin to debug this.

halamalagarli commented 1 year ago

Hello again,

With the same config file but running Python 3.10 on the machine with the exporter I get a similar but different trace files

2022-11-15 15:41:31,282 INFO:Starting web server on port :9272 2022-11-15 15:42:25,620 INFO:Start collecting metrics from vcenter1.campus.bath.ac.uk 2022-11-15 15:42:25,621 INFO:Starting host metrics collection 2022-11-15 15:42:25,621 INFO:Fetching vim.HostSystem inventory 2022-11-15 15:42:25,624 INFO:Retrieving service instance content 2022-11-15 15:42:25,631 INFO:START: _vmware_get_host_perf_manager_metrics 2022-11-15 15:42:28,871 ERROR:Caught vmodl fault: Cannot complete login due to an incorrect user name or password. 2022-11-15 15:42:28,874 ERROR:Traceback (most recent call last): File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 1937, in _async_render_GET yield self.generate_latest_metrics(request) twisted.internet.defer.FirstError: FirstError[#0, [Failure instance: Traceback: <class 'twisted.internet.defer.FirstError'>: FirstError[#0, [Failure instance: Traceback: <class 'AttributeError'>: 'NoneType' object has no attribute 'RetrieveContent' /var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py:735:errback /var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py:798:_startRunCallbacks /var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py:892:_runCallbacks /var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py:1792:gotResult --- --- /var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py:1693:_inlineCallbacks /var/tmp/.venv/lib/python3.10/site-packages/twisted/python/failure.py:518:throwExceptionIntoGenerator /var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py:707:host_system_inventory /var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py:1693:_inlineCallbacks /var/tmp/.venv/lib/python3.10/site-packages/twisted/python/failure.py:518:throwExceptionIntoGenerator /var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py:602:batch_fetch_properties /var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py:1697:_inlineCallbacks /var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py:594:content ]] --- --- /var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py:1642:_vmware_get_hosts /var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/defer.py:100:parallelize ]]

Unhandled error in Deferred:

Traceback (most recent call last): File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 735, in errback self._startRunCallbacks(fail) File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks self._runCallbacks() File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1792, in gotResult _inlineCallbacks(r, gen, status, context) --- --- File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks result = context.run( File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 707, in host_system_inventory host_systems = yield self.batch_fetch_properties( File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks result = context.run( File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 602, in batch_fetch_properties content = yield self.content File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks result = context.run(gen.send, result) File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 594, in content connection.RetrieveContent builtins.AttributeError: 'NoneType' object has no attribute 'RetrieveContent'

Unhandled error in Deferred:

Traceback (most recent call last): File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 735, in errback self._startRunCallbacks(fail) File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks self._runCallbacks() File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1792, in gotResult _inlineCallbacks(r, gen, status, context) --- --- File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks result = context.run( File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 707, in host_system_inventory host_systems = yield self.batch_fetch_properties( File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks result = context.run( File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 602, in batch_fetch_properties content = yield self.content File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks result = context.run(gen.send, result) File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 594, in content connection.RetrieveContent builtins.AttributeError: 'NoneType' object has no attribute 'RetrieveContent'

Unhandled error in Deferred:

Traceback (most recent call last): File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 735, in errback self._startRunCallbacks(fail) File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks self._runCallbacks() File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1792, in gotResult _inlineCallbacks(r, gen, status, context) --- --- File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks result = context.run( File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 1038, in host_labels dcs = yield self.datacenter_inventory File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks result = context.run( File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 970, in datacenter_inventory content = yield self.content File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks result = context.run(gen.send, result) File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 594, in content connection.RetrieveContent builtins.AttributeError: 'NoneType' object has no attribute 'RetrieveContent'

Unhandled error in Deferred:

Traceback (most recent call last): File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 735, in errback self._startRunCallbacks(fail) File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 798, in _startRunCallbacks self._runCallbacks() File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1792, in gotResult _inlineCallbacks(r, gen, status, context) --- --- File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks result = context.run( File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 1138, in counter_ids content = yield self.content File "/var/tmp/.venv/lib/python3.10/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks result = context.run(gen.send, result) File "/var/tmp/.venv/lib/python3.10/site-packages/vmware_exporter/vmware_exporter.py", line 594, in content connection.RetrieveContent builtins.AttributeError: 'NoneType' object has no attribute 'RetrieveContent'

bilbof commented 1 year ago

2022-11-15 15:42:28,871 ERROR:Caught vmodl fault: Cannot complete login due to an incorrect user name or password.

You've provided invalid credentials. The exporter requires read-only access

shrady11 commented 1 year ago

I could confirm the credentials are correct; but it still fails with "Caught vmodl fault: Cannot complete login due to an incorrect user name or password."

Samoht11 commented 1 year ago

I could confirm the credentials are correct; but it still fails with "Caught vmodl fault: Cannot complete login due to an incorrect user name or password."

I had the same, I had to include suffix @vsphere.local , so vsphere_user: "user@vsphere.local"