Closed spamajor closed 1 year ago
@spamajor, thanks for letting me know. I'm pretty sure I had it running on 2.9x. I have now upgraded to the 3.00 beta to get prepare for the next version. You said that PyYaml install fails, do you have any error messages or something that might explain why?
Thanx in advance and here you go (clean install on sd-card, no incremental update whatsoever)
Starting random number generator daemon.
Configuring python3-xmlrpc.
Configuring python3-pip.
Pip install module dependencies
Collecting PyYAML==6.0
Downloading PyYAML-6.0.tar.gz (124 kB)
|████████████████████████████████| 124 kB 8.9 kB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", lin e 186, in _main
status = self.run(options, args)
File "/usr/lib/python3.8/site-packages/pip/_internal/commands/install.py", lin e 331, in run
resolver.resolve(requirement_set)
File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
discovered_reqs.extend(self._resolve_one(requirement_set, req))
File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
abstract_dist = self.preparer.prepare_linked_requirement(req)
File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", l ine 515, in prepare_linked_requirement
abstract_dist = _get_prepared_distribution(
File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", l ine 95, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(finder, build_isolation)
File "/usr/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata
self._setup_isolation(finder)
File "/usr/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation
reqs = backend.get_requires_for_build_wheel()
File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 1 51, in get_requires_for_build_wheel
return self._call_hook('get_requires_for_build_wheel', {
File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 2 55, in _call_hook
raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last ):
File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", lin e 63, in _build_backend
obj = import_module(mod_path)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "
And when I install PyYAML 6 manually I end up with this:
A fikkering screen constant between these two screenshots. Updates to VRM stops completey, realtime keeps working.
I do have to make a decision, either stick with YAML or ditch it in favour of JSON or Python classes. I can't have PyYaml corrupting devices. I am hoping that Venus 3.0 might address this problem some how.
Can you do a
python --version
and
python -m pip --version
on your device and tell me what version of python your Venus 2.92 has?
root@raspberrypi2:/data/drivers/dbus-mqtt-devices-0.5.1/bin# python --version Python 3.8.13
root@raspberrypi2:/data/drivers/dbus-mqtt-devices-0.5.1/bin# python -m pip --version pip 20.0.2 from /usr/lib/python3.8/site-packages/pip (python 3.8)
Thanks, that's the same as my setup. Still puzzled.
I have no clue where to start, tried new SD, version from May, 2.92. I behaves quite erratic now,
If you Google the actual error "ModuleNotFoundError: No module named 'dataclasses'" the suggestion is to install dataclasses. On Venus you'd need to do
python -m pip install dataclasses
Bummer, tried that already. my workaround was this, from the CCGX installer ->
wget -qO- https://files.pythonhosted.org/packages/36/2b/61d51a2c4f25ef062ae3f74576b01638bebad5e045f747ff12643df63844/PyYAML-6.0.tar.gz | tar xvz -C $BASE/ext cd $BASE/ext/PyYAML-6.0 python setup.py --without-libyaml install
After installing the manual PyYaml your script works, the device is showing up and is very responsive.
ERROR: Exception:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
status = self.run(options, args)
File "/usr/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 331, in run
resolver.resolve(requirement_set)
File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
discovered_reqs.extend(self._resolve_one(requirement_set, req))
File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
abstract_dist = self.preparer.prepare_linked_requirement(req)
File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement
abstract_dist = _get_prepared_distribution(
File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 95, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(finder, build_isolation)
File "/usr/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata
self._setup_isolation(finder)
File "/usr/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation
reqs = backend.get_requires_for_build_wheel()
File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 151, in get_requires_for_build_wheel
return self._call_hook('get_requires_for_build_wheel', {
File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 255, in _call_hook
raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 63, in _build_backend
obj = import_module(mod_path)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "
Would you be brave enough to try the Venus 3.0 beta? Is this for a production system?
Ha I started this adventure on 3.00~15 allas. btw. If you got it working and you have a Raspi2 too, shall i try your sd-image?
Maybe a clue, if I only forward the values for Power and Voltage it keeps on working. If I connector more it fails directly. I looking into it over the next days. Regards!.
This driver only handles the registration phase. Once you are sending data, your are using Victron's own dbus-Mqtt interface directly.
I don't understand how you are able to send data to the correct topics if you are not able to register the device. The deviceinstance id that you get from device registration is the important part of the topic name.
Hi, the bottomleft nodes subscribe to a topic and report the
But I think those topics must include the devicenstance. Without that you could be updating the wrong parameters on the dbus. Do you wait for device registration to complete before sending data values?
If you got it working and you have a Raspi2 too, shall i try your sd-image?
Sorry, I have raspberry pi 4. It's just a fresh install of Venus OS
But I think those topics must include the devicenstance. Without that you could be updating the wrong parameters on the dbus. Do you wait for device registration to complete before sending data values?
Yes, I wait for a reply: device/xxx/DBus : msg.payload : Object object portalId: "xxxx" deviceInstance: object domoticz: "xxxx"
The instance + portalid are part of the topic after te reply. As specified.
Bummer, tried that already. my workaround was this, from the CCGX installer ->
wget -qO- https://files.pythonhosted.org/packages/36/2b/61d51a2c4f25ef062ae3f74576b01638bebad5e045f747ff12643df63844/PyYAML-6.0.tar.gz | tar xvz -C $BASE/ext cd $BASE/ext/PyYAML-6.0 python setup.py --without-libyaml install
After installing the manual PyYaml your script works, the device is showing up and is very responsive.
@spamajor I tried to shorten your pyyaml install command to the following:
pip install pyyaml --global-option=--without-libyaml
But strangely I still get the same ModuleNotFoundError: No module named 'dataclasses'
error.
Following your method worked.
Hi @mtucker502 Many thanks indeed for helping here. Can you confirm whether pip install pyyaml --global-option=--without-libyaml did or did not work for you. It's not clear from your comment.
It still does not work.
Part of it has been solved for me. One of the nodes sent a string instead of int / float. So that broke everything. Thanx for the patience ...
PyYalm via Pip keeps complaining moduleNotFoundError: No module named 'dataclasses' error but the manual method works fine the 2nd and 3th time.
Glad you found the source of your problem. I'm hoping that the pip issue will get resolved in Venus 3.
Can we close this issue now?
Bummer, tried that already. my workaround was this, from the CCGX installer ->
wget -qO- https://files.pythonhosted.org/packages/36/2b/61d51a2c4f25ef062ae3f74576b01638bebad5e045f747ff12643df63844/PyYAML-6.0.tar.gz | tar xvz -C $BASE/ext cd $BASE/ext/PyYAML-6.0 python setup.py --without-libyaml install
After installing the manual PyYaml your script works, the device is showing up and is very responsive.
ERROR: Exception: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main status = self.run(options, args) File "/usr/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 331, in run resolver.resolve(requirement_set) File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve discovered_reqs.extend(self._resolve_one(requirement_set, req)) File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one abstract_dist = self._get_abstract_dist_for(req_to_install) File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for abstract_dist = self.preparer.prepare_linked_requirement(req) File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement abstract_dist = _get_prepared_distribution( File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 95, in _get_prepared_distribution abstract_dist.prepare_distribution_metadata(finder, build_isolation) File "/usr/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata self._setup_isolation(finder) File "/usr/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation reqs = backend.get_requires_for_build_wheel() File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 151, in get_requires_for_build_wheel return self._call_hook('get_requires_for_build_wheel', { File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 255, in _call_hook raise BackendUnavailable(data.get('traceback', '')) pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 63, in _build_backend obj = import_module(mod_path) File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 961, in _find_and_load_unlocked File "", line 219, in _call_with_frames_removed File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "", line 843, in exec_module File "", line 219, in _call_with_frames_removed File "/var/volatile/tmp/pip-build-env-82tmjt30/overlay/lib/python3.8/site-packages/setuptools/init.py", line 16, in import setuptools.version File "/var/volatile/tmp/pip-build-env-82tmjt30/overlay/lib/python3.8/site-packages/setuptools/version.py", line 1, in import pkg_resources File "/var/volatile/tmp/pip-build-env-82tmjt30/overlay/lib/python3.8/site-packages/pkg_resources/init.py", line 87, in import('pkg_resources.extern.packaging.requirements') File "/var/volatile/tmp/pip-build-env-82tmjt30/overlay/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 8, in from ._parser import parse_requirement File "/var/volatile/tmp/pip-build-env-82tmjt30/overlay/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_parser.py", line 10, in from ._tokenizer import DEFAULT_RULES, Tokenizer File "/var/volatile/tmp/pip-build-env-82tmjt30/overlay/lib/python3.8/site-packages/pkg_resources/_vendor/packaging/_tokenizer.py", line 3, in from dataclasses import dataclass ModuleNotFoundError: No module named 'dataclasses'
@spamajor Please see https://github.com/freakent/dbus-mqtt-devices/blob/python-modules/bin/setup.sh for an updated setup.sh script to address the missing dataclasses issue. I do not recommend using the pip install dataclasses
method.
I had the same errors while installing like @spamajor reported. I did this to get it work:
wget -qO- https://files.pythonhosted.org/packages/36/2b/61d51a2c4f25ef062ae3f74576b01638bebad5e045f747ff12643df63844/PyYAML-6.0.tar.gz | tar xvz -C /data/drivers/dbus-mqtt-devices-0.5.1/ext
cd /data/drivers/dbus-mqtt-devices-0.5.1/ext/PyYAML-6.0
python setup.py --without-libyaml install
python -m pip install dataclasses
reboot
Now the log shows me:
@40000000644e46611aeac904 *** starting dbus-mqtt-devices ***
@40000000644e46642098c3c4 -------- dbus_mqtt_devices, v0.5.1 is starting up --------
@40000000644e46642098e6ec INFO:logger:Loglevel set to INFO
@40000000644e466422533564 INFO:device_manager:Using portalId c0619ab2e89d
@40000000644e46650dcc9d9c INFO:mqtt_gobject_bridge:[Init] Connecting to local broker
@40000000644e46651129a37c INFO:device_manager:[Connected] Result code 0
Please try this version of the setup script. https://github.com/freakent/dbus-mqtt-devices/blob/python-modules/bin/setup.sh
I have a bigger problem now. The mqtt service stopped working and I have no idea what happened. I already tried to delete your script and did a firmware downgrade to 2.92 but it didn't helped. Do you have any tip for me what to check?
scroll down towards the bottom of the readme, there are troubleshooting tips and instructions on how to check the log file
The updated setup.sh script is now in the latest release v0.6.2.
After downgrading to 2.92 and a second reboot the mqtt service worked normal again. I tried your updated version of the setup script, and it worked without errors now. Checking rc.local looks good, too. Checking the status says:
root@einstein:/data/drivers# svstat /service/dbus-mqtt-devices
/service/dbus-mqtt-devices: up (pid 10867) 156 seconds
The log also shows no errors this time:
root@einstein:/data/drivers# more /var/log/dbus-mqtt-devices/current
@40000000644e7bf03a961d94 *** CCGX booted (0) ***
@40000000644e7ec71cb2fe14 *** CCGX booted (0) ***
@40000000644e80ab1490bf64 *** CCGX booted (0) ***
@40000000644e8eea01113f2c *** starting dbus-mqtt-devices ***
@40000000644e8eec0ac46e7c -------- dbus_mqtt_devices, v0.5.1 is starting up --------
@40000000644e8eec0ac491a4 INFO:logger:Loglevel set to INFO
@40000000644e8eec0cb46b24 INFO:device_manager:Using portalId c0619ab2e89d
@40000000644e8eec3512d484 INFO:mqtt_gobject_bridge:[Init] Connecting to local broker
@40000000644e8eec3a8b9644 INFO:device_manager:[Connected] Result code 0
Than I send the payload
{ "clientId": "fe001", "connected": 1, "version": "v1.0", "services": {"t1": "temperature"} }
to the topic
device/fe001/Status
and received this in the log:
@40000000644e908a13a0e8f4 INFO:device_manager:Received device status message {'clientId': 'fe001', 'connected': 1, 'version': 'v1.0', 'services': {'t1': 'temperature'}}
@40000000644e908a13cb583c INFO:device:**** Registering device: fe001, services: {'t1': 'temperature'} ****
@40000000644e908a13d7009c INFO:device:Registering Service temperature for client fe001
@40000000644e908a13e24f24 INFO:device_service_config:About to open config file
@40000000644e908b01079a6c INFO:device_service:Registering service temperature for client fe001 at path com.victronenergy.temperature.mqtt_fe001_t1
@40000000644e908b0498048c INFO:settingsdevice:Setting /Settings/Devices/mqtt_fe001_t1/ClassAndVrmInstance does not exist yet or must be adjusted
@40000000644e908b0921b32c INFO:vedbus:registered ourselves on D-Bus as com.victronenergy.temperature.mqtt_fe001_t1
@40000000644e908b0a96693c INFO:device_service:Registered Service com.victronenergy.temperature.mqtt_fe001_t1 under DeviceInstance 1
@40000000644e908b0adeb50c INFO:device_manager:publish {'portalId': 'c0619ab2e89d', 'deviceInstance': {'t1': '1'}} to device/fe001/DBus, status is 0
@40000000644e908b0b0e5474 INFO:device_manager:Received device status message {'clientId': 'fe001', 'connected': 1, 'version': 'v1.0', 'services': {'t1': 'temperature'}}
@40000000644e908b0b4060f4 INFO:device_manager:publish {'portalId': 'c0619ab2e89d', 'deviceInstance': {'t1': '1'}} to device/fe001/DBus, status is 0
On topic
device/fe001/DBus
I received back:
{"portalId": "c0619ab2e89d", "deviceInstance": {"t1": "1"}}
Looks all good for me. A new temperature sensor now shows up in the Remote Console. The old name of the sensor was restored, so there must be any data kept.
At this point I restarted the cerbo and after the reboot mqtt doesn't send any data... I send my keepalive to R/c0619ab2e89d/keepalive every 55 seconds and the cerbo doesn't give me any data back. The log just says:
@40000000644e954c3acd69d4 *** CCGX booted (0) ***
@40000000644e957433994444 *** starting dbus-mqtt-devices ***
@40000000644e957e35190e44 -------- dbus_mqtt_devices, v0.5.1 is starting up --------
@40000000644e957e3519316c INFO:logger:Loglevel set to INFO
@40000000644e957f0056e424 INFO:device_manager:Using portalId c0619ab2e89d
@40000000644e95811089f87c INFO:mqtt_gobject_bridge:[Init] Connecting to local broker
@40000000644e95812283904c INFO:device_manager:[Connected] Result code 0
Do you have any idea???
Thanks for sending all the logs as you have done, that makes it much easier to debug.
You said
At this point I restarted the cerbo and after the reboot mqtt doesn't send any data... I send my keepalive to R/c0619ab2e89d/keepalive every 55 seconds and the cerbo doesn't give me any data back.
What do you mean by mqtt doesn't send any data? What are you expecting to happen?
By the way, looking at your logs it looks like you are registering the same device twice. It's not a problem, but it is unnecessary.
I connected the cerbo to my smart home system via mqtt and for debugging I use MQTT Explorer. Now at the moment the communication is dead. No mqtt messages at all. I allready removed the line from rc.local and the link it creates /service/dbus-mqtt-devices, too. after reboot nothing changed, no mqtt messages at all...
Is the display still working? Can you still ssh to the GX device?
Yes the remote console and ssh works normal
If you look under Settings > Services what do you see?
I already tried to disable and enable mqtt again. No change
Can you send a screen shot of what you do see in MQTT Explorer?
I subscribed the topics vebus, battery and pvinverter, but the values are not getting updated. only the $SYS values come in every few seconds.
OK, so you are getting MQTT messageds, otherwise MQTT Explorer would be empty.
Can try publishing an empty message to R/co619ab2e89d/system/0/Serial?
I cleared the MQTT Explorer and published an empty message to R/c0619ab2e89d/system/0/Serial but nothing changed.
Check the dbus-mqtt logs. $ more /var/log/dbus-mqtt/current
I attached the file: current.txt
For me it looks like the dbus-mqtt in a bootloop and doesn't start correct...
I agree. It's complaining about a dbus service of "/". What's in your services.yml ? Are you using the standard version of a customised version?
Where do I find this file? I haven't customized anything. But I'm running another script for me ET340 simulation https://github.com/mitchese/shm-et340
I had the same issue at the beginning after installing the script from mitchese and this was the solution for the problem: https://community.victronenergy.com/questions/157757/cerbox-gx-mqtt-no-topics-exept-sys.html
maybe it helps, finding the issue
Run dbus-spy, that will show you exactly what is on the dbus that dbus-mqtt is reading from.
$ dbus-spy
This is the output from dbus-spy:
Yes I'm running shm-et340 at the same time now. I need it to import the data from my SMA Sunny Home Manager as energy meter. Can this cause the problems?
Hi, on my Raspberry Pi PyYalm fails on install and with workaround the storage + VRM updates breaks on ver 2.92. See screetshot: