Adapter does not support requesting another target address

[chrizzz86]

Describe the bug
I tried to request some parameters from different target than the default one (default target address 0), e.g. 710!8.

Expected behavior
It would be great to consider also the "!" sign extension to address also other targets. Basiclaly, the converter should support this (e.g. /JQ=710!8,710!7), but it seems that the current implementation only reads all possible categories and their parameters. The requested parameter is then compared against this, which fails due to the addition of the target address (710 would be valid instead of 710!8).

Screenshots & Logfiles

Versions:

• Adapter version: v0.3.1
• JS-Controller version: v5.0.12
• Node version: v16.20.2
• Operating system: Armbian 23.8.1 Bullseye with Linux 5.15.93-rockchip64

[hacki11]

The adapter fetches the parameters using a bulk operation due to performance reasons. Therefor I need to find the ids in the response again. If the address is not within the response I won't find it. Do you know if it is somehow in the response object?

[chrizzz86]

Yes, the response does also contain the destination.

e.g. Request: {ip}/JQ=710!8,710!7 Response: { "710": { "name": "Komfortsollwert", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "8", "error": 0, "value": "20.0", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" }, "710": { "name": "Komfortsollwert", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "7", "error": 0, "value": "21.5", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" } }

[hacki11]

Interesting, in my responses it is not available. Do you know if this is only present since a specific version or if you have more than one target? What is the value if you don't specify an address?

[chrizzz86]

I don't know, but I would expect that the JSON response ist dependent of the software version. I think I'm using the most recent software version of BSB LAN Adapter.

Hardware: ESP32 Version: 3.3.2-20231115201023 Bus system: LPB

[hacki11]

The adapter is responding with a duplicate key for this object array. While it is valid JSON parsing to an object will be a problem for the library as one of the entries getting lost. Thats also what the browser does, in Firefox you could see only one 710 object in the JSON view: (Request: {ip}/JQ=710!8,710!7) But two in the RAW view: Depending on the parser you use, the behavior can be difficult.

Perhaps @dukess or @fredlcore has some statements regarding this. For parameters with .x it is different, here you get back the .x if x > 0. So if you request

• 20000.0
• 20000.1
• 20000.2 you get the following array: "20000": { ... } // .0 omitted! "20000.1": { ... } "20000.2": { ... }

But for

• 710!7
• 710!8 you get: "710": { ... destination: 7 } "710": { ... destination: 8 }

The strategy seems a bit inconsistent here. While the first one is easy to implement in the adapter, just omitting .0, too, the latter is hard if you only get one object into your library. Also the differentiation between the default address "710" without address and one with address is hard because I'm not aware of the default address. So the correct matching can not be done without getting the default address first which should be avoided to use the API.

[fredlcore]

You set the default destination address in BSB_LAN_config.h, and it's usually 0. If you call a parameter with ! that means you are temporarily diverting from the default destination for this one call. I'm a bit surprised the destiantion doesn't show up in the screenshot above, but you are mentioning it below? The diversion from the default destination address using ! is something different than the parameters with a decimal point which differentiate between normal parameters and those which carry more than one datapoint. So, I'm not sure what would be helping here, especially since nothing should break compatibility. We have the calls to non-default devices using the ! for I think half a decade now, although offering this via JSON happened around two years ago. Anything I can do to make this problem go away is fine with me as long as it doesn't introduce new problems for established installations.

[hacki11]

Thank you for your feedback @fredlcore. I'm using a 2.2.x version where I guess the destination is not yet implemented but saw it on @chrizzz86 response so I take it as granted. I wonder if anyone is able to use the duplicate key in such a response. As I request in bulk mode to save resources I'd try to avoid collecting one by one for this use case.

I guess what I would have expected than was something like 20000.1!7 if we want to see both syntaxes in one example.

So 710 as response for !0 and 710!x for x > 0 would be consistent I guess

[chrizzz86]

Thank you for your interest and the responses.

I'm a little bit surprised that I'm the only user who wants to address components in the LPB that do not correspond to the default address. In my case, the main component of Weishaupt WRS-CPU-B2/E does not provide all parameters. But, by addressing the extension module WRS-EM temporary, I can then query specific information about my heating circuit. It would be great to have such feature available for this adapter.

Please let me know, if I can support to test, etc...

[fredlcore]

Not sure if I understand what you mean with "both syntaxes in one example". Also, 710 for !0 (or whatever your default destination address is) and 710!x for x>0 (or rather x<>default destination address) is what is implemented. 710.1!x would also work if your heater has more than one data point under 710 and you query it from a device other than the default destination. So I'm not sure what would have to change here.

Other than that, there is hardly any scenario why you would want/need to query the same parameter number from more than one device. In an LPB setting, the master (usually device ID 0) has the authority over all other devices in a segment. So while a slave might still offer parameter 710, it is of no relevance as only 710 from the master determines the flow temperature. Querying slaves only makes sense if you want to know device-specific values such as boiler temperature which may be different from flow temperature which the master "knows". But still you would not have doublettes. These would only come into effect in larger scenarios where you have a cascade of several boilers where you want to read the boiler temperature from each one of them. Or you have a setup with several segments and want to query several masters. But that would be an even larger setup. Again, if there's a way for the JSON to make it clearer, then I'm fine changing that. But I'm not sure that deviating from a numerical value towards one such as "710!1" (in addition to the destination attribute) would help...

[hacki11]

You would wonder yourself what features are on the market which aren't used at all (unrelated to bsblan)

I'm open to support this in the adapter, let's see what options we have.

[fredlcore]

@chrizzz86: No, you're not the only one. I also do this in my setup of course. But not for the same parameter on several controllers on my LPB. That's why I've never encountered @hacki11's problem. But as I said, there may be a few cases in rather large setups where this problem may occur, and if you (@hacki11) can provide a solution that is not breaking compatibility, I'm all in.

[chrizzz86]

Maybe there's a confusion regarding my request. I also do not request the same parameter on two devices in parallel. I just wanted to indicate that also on a bulk operation like {ip}/JQ=710!8,710!7 (that's at least what I understand by bulk operation), the response contains both values that can be distinguished from each other: Value from destination 7 and different value from destination 8.

[fredlcore]

Yes, with the destination attribute, they can be distinguished, but I understood @hacki11's comment that because both look the same on the root level, it would appear as one object. But I just realized that this is not an "issue" in my repo, but in @hacki11's, so I guess I'll just wait until further notice ;)...

[hacki11]

Ok, only accessing a parameter from a single device, yet individually addressable should be easy to implement. I wasn't aware of this legal limitation.

Regarding the "duplicate key" in the JSON response I want to say, that while it is perfectly legal in the JSON specification it is not supported by most common JSON parser in Java, JavaScript, Python, etc. They all will only take the last occurrence of a key dropping the objects before during deserialization of the JSON to an object. (One Source: https://stackoverflow.com/a/23195243/3342853)

So if we only stick with the given use case we are fine - I will validate the configuration to only allow requesting unique parameters to make sure the adapter won't run into such an issue.

[hacki11]

@chrizzz86 Can you install the development version from this branch? https://github.com/hacki11/ioBroker.bsblan/tree/different-address I added the support to allow accessing param!x while param is used only once. It won't show the address within the iobroker object tree. I would recommend to delete the existing object in iobroker after updating.

Let me know if there are any warnings or errors in the log.

[fredlcore]

One more remark: /JQ was initially meant to receive data via a POST request where the information about the destination would be sent as an attribute and also returned likewise. The fact that it works also via the URL using the exclamation mark was not intended, but works due to BSB-LAN's internal handling of URL commands.

[hacki11]

Thank you for the update, this gives a new perspective on this topic. I'm glad we could hopefully solve it now easily just having a unique parameter :) Let's see what the tests will show

[chrizzz86]

Hi hacki11,

thanks for your fast response and implementation. Unfortunately, it seems not to be working as expected:

My config looks like this:

Based on the log, I installed commit e686fcb653b86ea7a2807afc3c5bbbd69639c781, which should be the right one?

Best regards Christian

[hacki11]

Unfortunately, you got the wrong feature branch, please take that one: https://github.com/hacki11/ioBroker.bsblan/tree/different-address

[chrizzz86]

Sorry, my fault...

However, it still doesn't seem to be working properly. It seems that some other parameters are also requested from destination address 8.

Log: 2024-03-11 20:59:55.873 - debug: bsblan.0 (3410113) /JQ Response: {"81":{"name":"Wartungsmeldung","dataType_name":"ENUM","dataType_family":"ENUM","destination":"8","error":7,"value":"","desc":"","dataType":1,"readonly":1,"readwrite":1,"unit":""},"110":{"name":"Aussentemperaturfühler lokal","dataType_name":"TEMP","dataType_family":"VALS","destination":"0","error":0,"value":"5.1","desc":"","precision":0.1,"dataType":0,"readonly":1,"readwrite":1,"unit":"°C"},"115":{"name":"Kesseltemperatur-Istwert","dataType_name":"TEMP","dataType_family":"VALS","destination":"0","error":0,"value":"61.4","desc":"","precision":0.1,"dataType":0,"readonly":1,"readwrite":1,"unit":"°C"},"116":{"name":"Vorlauftemperatur","dataType_name":"TEMP","dataType_family":"VALS","destination":"8","error":0,"value":"36.9","desc":"","precision":0.1,"dataType":0,"readonly":1,"readwrite":1,"unit":"°C"},"118":{"name":"Warmwassertemperatur-Istwert","dataType_name":"TEMP","dataType_family":"VALS","destination":"8","error":7,"value":"","desc":"","precision":0.1,"dataType":0,"readonly":1,"readwrite":1,"unit":"°C"},"120":{"name":"Kaskaden-Vorlauftemperatur-Istwert","dataType_name":"TEMP","dataType_family":"VALS","destination":"8","error":7,"value":"","desc":"","precision":0.1,"dataType":0,"readonly":1,"readwrite":1,"unit":"°C"}}

[chrizzz86]

Update: When I request the values manually: {IP-ADDRESS}/JQ=81,110,115,116!8,120 -> the result is also not correct! I guess that's based on the BSB LAN adapter software.

I changed my config to this (added zero destination):

Which is now working properly, great!

[hacki11]

Haha, I was just writing to ask you to try it with the API itself.

[chrizzz86]

Great, thanks for your fast support and implementation. I would close this issue with that greate result!

[hacki11]

Can you incrementally check when this issue is rising in the api?

JQ=81,116!8 JQ=116!8,81 and so on until we have the simplest example?

[hacki11]

@fredlcore Shall we open an issue on bsb_lan for this behavior? Currently we see that if we request multiple parameters, but one with a non-default destination with a JQ query the destinations get mixed up. If all parameters are requested with their destination, even for default destination it works. Perhaps it is related to the set_temp_destination call which is perhaps not triggered correctly in this use case.

You said, it was never really implemented, so it is currently undefined what happens. But I guess it would be great if it could be fixed somehow.

[chrizzz86]

After some more investigation regarding the API... Perhaps the error is simply that the specific address 8 is not being reset.

If I set the specific address 8 at the first position and the second one to zero destination address, it seems to work as expected.

Request: {IP-ADDRESS}/JQ=116!8,81!0,110,115,120 Response: { "116": { "name": "Vorlauftemperatur", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "8", "error": 0, "value": "42.5", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" }, "81": { "name": "Wartungsmeldung", "dataType_name": "ENUM", "dataType_family": "ENUM", "destination": "0", "error": 0, "value": "1", "desc": "1:Brennerbetriebsstunden überschritten", "dataType": 1, "readonly": 1, "readwrite": 1, "unit": "" }, "110": { "name": "Aussentemperaturfühler lokal", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "0", "error": 0, "value": "4.2", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" }, "115": { "name": "Kesseltemperatur-Istwert", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "0", "error": 0, "value": "53.5", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" }, "120": { "name": "Kaskaden-Vorlauftemperatur-Istwert", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "0", "error": 0, "value": "53.5", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" } }

If I do not add the zero destination address, all values seems to be requested for destination address 8. Request: {IP-ADDRESS}/JQ=116!8,81,110,115,120 Response: { "116": { "name": "Vorlauftemperatur", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "8", "error": 0, "value": "42.5", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" }, "81": { "name": "Wartungsmeldung", "dataType_name": "ENUM", "dataType_family": "ENUM", "destination": "8", "error": 7, "value": "", "desc": "", "dataType": 1, "readonly": 1, "readwrite": 1, "unit": "" }, "110": { "name": "Aussentemperaturfühler lokal", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "8", "error": 0, "value": "---", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" }, "115": { "name": "Kesseltemperatur-Istwert", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "8", "error": 7, "value": "", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" }, "120": { "name": "Kaskaden-Vorlauftemperatur-Istwert", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "8", "error": 7, "value": "", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" } }

I'm a bit confused because the behavior yesterday was different when using the adapter. In that case also the first request for parameter "81" started with destination address 8 but I don't know how the requests are ordered by the adapter.

Sorry, my fault...

However, it still doesn't seem to be working properly. It seems that some other parameters are also requested from destination address 8.

Log: 2024-03-11 20:59:55.873 - debug: bsblan.0 (3410113) /JQ Response: {"81":{"name":"Wartungsmeldung","dataType_name":"ENUM","dataType_family":"ENUM","destination":"8","error":7,"value":"","desc":"","dataType":1,"readonly":1,"readwrite":1,"unit":""},"110":{"name":"Aussentemperaturfühler lokal","dataType_name":"TEMP","dataType_family":"VALS","destination":"0","error":0,"value":"5.1","desc":"","precision":0.1,"dataType":0,"readonly":1,"readwrite":1,"unit":"°C"},"115":{"name":"Kesseltemperatur-Istwert","dataType_name":"TEMP","dataType_family":"VALS","destination":"0","error":0,"value":"61.4","desc":"","precision":0.1,"dataType":0,"readonly":1,"readwrite":1,"unit":"°C"},"116":{"name":"Vorlauftemperatur","dataType_name":"TEMP","dataType_family":"VALS","destination":"8","error":0,"value":"36.9","desc":"","precision":0.1,"dataType":0,"readonly":1,"readwrite":1,"unit":"°C"},"118":{"name":"Warmwassertemperatur-Istwert","dataType_name":"TEMP","dataType_family":"VALS","destination":"8","error":7,"value":"","desc":"","precision":0.1,"dataType":0,"readonly":1,"readwrite":1,"unit":"°C"},"120":{"name":"Kaskaden-Vorlauftemperatur-Istwert","dataType_name":"TEMP","dataType_family":"VALS","destination":"8","error":7,"value":"","desc":"","precision":0.1,"dataType":0,"readonly":1,"readwrite":1,"unit":"°C"}}

[hacki11]

In the log there is the line "values found" and it is already exactly ordered like requested via the API.

Your observation confirms a bit my guess that bsblan does not reset the destination address if omitted after provided for one paramaeter.

[fredlcore]

What you describe is intended behaviour at least when querying a range of parameters using /x-y!z, however I thought that this is not the case when querying individual parameters using /a/b/c (maybe you can confirm this, @chrizzz86?). The reason for not switching back even though you want to query the same destination device, is that it would query the device family and variant for each parameter in between these calls which would add unnecessary load on the bus and also slow things down unnecessarily. Since I already said that calling /JQ via HTTP GET is not defined, the behaviour for comma selected parameters seems to be the same as for querying a range of parameters. TBH, I actually prefer it that way because it enables you to query even parameters that are not part of a range from a different destination. You just need to make sure (and document this behaviour) that when mixing destinations, you cannot assume a default destination once you have changed it temporarily for this call, so you have to explicitly switch back to the default destination if this is what you want.

[hacki11]

As I don't know the default address in the adapter nor do I want to know it, I can not switch back. I guess I will make one call per destination address. That way we can use it like it is. :)

[chrizzz86]

Ok, you're right. I was just wondering why the first parameter "81" did not respond with destination 0 address but it seems to be ordered at the end of the parameter collection. I agree with the assumption that the destination address is not reset.

[chrizzz86]

What you describe is intended behaviour at least when querying a range of parameters using /x-y!z, however I thought that this is not the case when querying individual parameters using /a/b/c (maybe you can confirm this, @chrizzz86?). The reason for not switching back even though you want to query the same destination device, is that it would query the device family and variant for each parameter in between these calls which would add unnecessary load on the bus and also slow things down unnecessarily. Since I already said that calling /JQ via HTTP GET is not defined, the behaviour for comma selected parameters seems to be the same as for querying a range of parameters. TBH, I actually prefer it that way because it enables you to query even parameters that are not part of a range from a different destination. You just need to make sure (and document this behaviour) that when mixing destinations, you cannot assume a default destination once you have changed it temporarily for this call, so you have to explicitly switch back to the default destination if this is what you want.

Let me explain with some examples:

JSON Response: Request: {IP}//JQ=116!8,81 "Wrong Behavior": All parameters are requested for destination address 8

Request: {IP}//JQ=116!8,81!0 "Right Behavior": Parameter 116 was requested for address 8, 81 for 0

HTML Response: Request: {IP}/116!8/81 "Right Behavior": Parameter 116 was requested for address 8, 81 for 0

The HTML request works as expected, without using the default destination address

[fredlcore]

So the undefined /JQ request is actually consistent and thus defined according to the range request when using the HTML response: /8000!1-8006/700 queries parameters 8000 to 8006 from device ID 1 and then parameter 700 from the default device. The comma separated parameters thus follow the range procedure in that they do not change the changed destination address. Any desired change needs to be explicitly called if it should happen in the same request.

[hacki11]

So I just need to put the parameters without a specific destination at the front of my query?

Example: 710,116!8,81,110!7 should result in the query like: 81, 710, 110!7, 116!8

That would be easy to implement and sounds also like a good solution.

[fredlcore]

Yes, if you can reorder it that way, that should do what you want - but better check first, I have no means to test this on real hardware at the moment...

[hacki11]

Me neither, I hope @chrizzz86 can do the tests :) If he can confirm it works like this I can provide an update of the implementation later

[chrizzz86]

Sure, I will check that

[chrizzz86]

Ok, if I got it right I tried to change my JSON request from... .../JQ=81,110,115,116!8,120 to "destination-sorted" request : /JQ=81,110,115,120,116!8 Response: { "81": { "name": "Wartungsmeldung", "dataType_name": "ENUM", "dataType_family": "ENUM", "destination": "0", "error": 0, "value": "1", "desc": "1:Brennerbetriebsstunden überschritten", "dataType": 1, "readonly": 1, "readwrite": 1, "unit": "" }, "110": { "name": "Aussentemperaturfühler lokal", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "0", "error": 0, "value": "5.1", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" }, "115": { "name": "Kesseltemperatur-Istwert", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "0", "error": 0, "value": "62.1", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" }, "120": { "name": "Kaskaden-Vorlauftemperatur-Istwert", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "0", "error": 0, "value": "62.1", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" }, "116": { "name": "Vorlauftemperatur", "dataType_name": "TEMP", "dataType_family": "VALS", "destination": "8", "error": 0, "value": "42.8", "desc": "", "precision": 0.1, "dataType": 0, "readonly": 1, "readwrite": 1, "unit": "°C" } }

Looks good and as expected! I would be also open to test it for the adapter, just let me know...

[hacki11]

Awesome! I'll start implementation this evening

[hacki11]

Here we go, sorted by id, address and destination. The log line shows it already sorted that way. https://github.com/hacki11/ioBroker.bsblan/tree/sort-destination

[chrizzz86]

Hello again, thanks for the udpate! Short answer: Works everything fine 👍 Long answer: Works everything fine 👍 (Please note: space means line break inside the code block...)

81 110 114!8 115 116!8 118 120

81!0 110!0 114!8 115!0 116!8 118!0 120!0

81,110,114!8,115 116!8 118 120

...and finally, the following fails as expected due to the duplicated key for parameter 116 81 110 114!8 115 116!8 116!7 118 120

[hacki11]

Awesome! Thank you for testing and also thank you @fredlcore for explaining everything bsb_lan related!

[fredlcore]

My pleasure - since I'd guess that up to now users in the three digit range are using your plugin (beaten only by HomeAssistant installations), I'm always happy to help!