RAW telegrams over TCP

[csowada]

Hello John,

I've created an eBUS java binding for openhab. Now I try to connect the ebusd deamon to my java binding. So I tried the TCP connection, but in my case I only need the the raw data. Is it possible to also receive the raw telegrams?

[john30]

not yet, but this could be added. how exactly would you like to have it on the wire then? if as raw bytes, this should rather be handled over a separate prot I'd say

[csowada]

i've done my first tests with "socat" to pipe the stout with the raw data to a secondary tcp port. This kind of data is enough for me. But you are right, if you would add this to the current tcp connection it would not work well with the raw send command. The answer of the send command is not asynchrone ? Or is my guess wrong? Is an additional tcp socket with just the raw communication (binary or as hex string) a good idea? So you can use the current tcp channel to control the ebusd and another to read the raw data.

Another idea would to allow async send with an id and a following response mit the id and result. Than we could also use the same tcp connection.

But I hope I'm not totally wrong with my ideas. But I thing it is a good idea to ask.

Thanks

[john30]

I think it would be better to have a possibility of using a higher level protocol where your client can issue a read/write by sending the necessary message part and be automatically informed of incoming messages, i.e. skipping all nonsense in between like the SYN bytes and incomplete/invalid messages (according to eBUS protocol and CRC check). This could be acomplished by adding a new command e.g. "direct" to the ebusd TCP port that turns the connection into direct send/receive mode. This could be done in human readable hex or also binary if you prefer. For hex a typical communication could look like this (sent data by your client prefixed with ">", received prefixed with "<" for clarity only):

>direct
<direct mode entered, stop with 'end'.
>08070400
<1008b5130304cd017f00
<08070400: 000ab54548503030032772016b 00
>35070400
<10feb505034a0100f4
<1008b5090329b801fd00
>35070400: timeout
...
>end
<direct mode stopped.

Some explanation:

1. client sends: "direct"
2. ebusd answers: "direct mode entered, stop with 'end'.
3. client wants to read ID of device 08 and sends: "08070400"
4. another message was just received by ebusd and ebusd sends: "1008b5130304cd017f00"
5. previously request ID message was answered, so ebusd sends: "08070400: 000ab54548503030032772016b 00", i.e. it repeats the desired message that was supposed to be sent and appends the response after a colon.
6. client wants to read ID of device 35 and sends: "35070400"
7. further messages are received while waiting for the ID message to be answered and resending the message due to repetition settings
8. more messages are received by ebusd in between
9. ebusd detects a timeout for the ID of device 35 and sends: "35070400: timeout"

The same exercise could of course be done by using a binary mode for everythign instead of hex. For this, the possible commands and results need to be assigned to a certain value, but I think the hex or human readable mode is also good to have. Of course it would also be possible to not use the sent part of the message as ID, but it would be easy this way. What do you think?

[csowada]

Hello John,

thank you for your time. I totally agree with your idea to only handle high level access. But I see one small problem with your approach. In that case I'm not able to query other information from the ebusd daemon like the "state" and "info" commands without switching the direct mode off and on again. In the "switch" time I could lose some telegrams. Or I'm wrong?! But if you include it to the current TCP port I would prefer the hex version to keep it align with the other commands. This evening I've worked on my binding to connect to your awesome ebusd daemon. I hope I'm ready for a first live test tomorrow.

[andig]

I‘m not really sure why all of this is necessary. Wouldn‘t it suffice to publish the raw messages on mqtt if enabled? Even raw queries could be done that way without inventing another protocol?

[disaster123]

That mode would be fantastic. I need that one as well for my ip symcon implementation. This direct mode would be a perfect fit: https://github.com/john30/ebusd/issues/269#issuecomment-473622588

[john30]

it is implemented now with slightly changed protocol. a typical conversation looks like this:

direct
direct mode started
1752b5040132 0a00280602010140070100
1008b5110102 050000c800c8
50b509030d3c00
50b509030d3c00:0101
1008b50903290700 050700250200
3150b509030d3c00 0101
1008b509040ed10000 00
stop
direct mode stopped

so basically a send request like the line 50b509030d3c00 above will always be answered with the same data plus colon and the result, where the result might be an error, done for successful write to a master, done broadcast for successfully sent broadcast, or the answer received from the slave.

Other messages seen on the bus are added with the master part and optional slave part separated by space, where the master part always starts with the master addess (in contrast to messages sent in this connection not starting with the master address!).

In direct mode, ebusd will also give a special help by sending "help" or "h", so a programmatic client should avoid that.

A message sent in direct mode will also appear a second time as seen message with the master address prefix.

So from a client perspective one could either only evaluate success of an issued message by waiting for the same prefix and checking the right side behind the first colon for error and then treat the potential answer from slave just like any other seen message, or directly look at the result (looking at the right side).

[csowada]

Wow, great news. Thank you for your quick help. I will implement that next days in my binding and give feedback.

[csowada]

Hello John,

one question. You write the raw data without CRC information. I would expect with direct mode the CRC is included? I know, you only send valid telegrams, so the CRC is not really needed.

Is it possible that the direct output is a bit delayed/buffered?

[andig]

I would assume that you get delays due to tcp window size?

Am 25.03.2019 um 21:29 schrieb csowada notifications@github.com:

Hello John,

one question. You write the raw data without CRC information. I would expect with direct mode the CRC is included? I know, you only send valid telegrams, so the CRC is not really needed.

Is it possible that the direct output is a bit delayed/buffered?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[john30]

since the CRC and ACK/NACK are part of the lower protocol, these are excluded. Yes, the data is buffered.