CAN-J1939 maximum message size is limited e.g. 4193975 bytes

[josemic]

This is a follow-up of part of #164.

I am sending static J1939 messages of the size 4193975. This is working. However if I send messagesc>= 4193976 bytes I get the message "(Message too long)". The same applies, If I send 2 messages at the same time from2 addresses to two different addresses.

Note: It is stable and receives the correct message for messages of up to 4193975 bytes.

Thus there are the following issues:

• It is impossible on my ubuntu PC with kernel 5.4 to send up to (2²⁴-1)*7= 117440505 bytes, which is the maximum size of ETP messages.
• There seems to be a fixed maximum message size of up to 4193975 bytes.
• Up to that maximum size the number of sent bytes and the return values of the sent bytes from the send function match

From #164:

Isobus as far as I know supports extended transport protocol with (2²⁴-1)*7= 117440505 Bytes

yes

How do I send and receive such long packages?

Just use the normal read()/write() send()/recv() and similar functions. However when sending you have to check the return value. If the functions returns less bytes than you have tried to send initially, you have to send the remaining bytes.

This is very strange. If you send one message, however only the first part of the message is sent and the next parts are sent in a second message and maybe more messages. This means when receiving a message you can not decode, because it is an incomplete message you have to wait for the next part, then combine those messages and repeat this process if the message is still not complete. This requires a very sophisticated message decoder.

I did not notice that the number of sent bytes and the return value, if not negative, does not match.

[marckleinebudde]

Just use the normal read()/write() send()/recv() and similar functions. However when sending you have to check the return value. If the functions returns less bytes than you have tried to send initially, you have to send the remaining bytes.

This is very strange. If you send one message, however only the first part of the message is sent and the next parts are sent in a second message and maybe more messages. This means when receiving a message you can not decode, because it is an incomplete message you have to wait for the next part, then combine those messages and repeat this process if the message is still not complete. This requires a very sophisticated message decoder.

A (E)TP message contains the total size in the fist packet. This is why you have to call send() with total size of your message. Callingsend() with the remaining bytes, doesn't start a new message, the data is queued for the current running message transfer.

However, there is currently no way to abort the currently transferred message.

[marckleinebudde]

I was neither able to send a single messages nor 2 messages at the same time (from2 different addresses to two different addresses).

Please give the commands to reproduce this issue.

Note: there are certain limitations to the j1939 (E)TP, you can only have one TP and one ETP transfer ongoing between two ECUs. A j1939 CAN frame is identified by src dst and pgn. TP uses two gpns, while ETP uses another pair.

[marckleinebudde]

It is impossible on my ubuntu PC with kernel 5.4 to send up to (2²⁴-1)*7= 117440505 bytes, which is the maximum size of ETP messages.

The sending side is not the problem, but the receiving side is. We have not optimized it for receiving large (E)TP messages. It depends on how much memory your kernel can allocate, which depends on the RAM of the system, the fragmentation of it and various other things.

We know that there is room to improve. As resources are limited, what's your use case for these large messages, or are you testing the stack in general?

[josemic]

Please give the commands to reproduce this issue.

I am using a early version of the eclipse Titan TTCN test port for SocketCAN titan.TestPorts.SocketCANasp supporting j1939 messages. I have not released the code yet. But soon I will do. I will leave here a note, when it is available and how to use it to test it.

Note: there are certain limitations to the j1939 (E)TP, you can only have one TP and one ETP transfer ongoing between two ECUs.

Ok. I am using only one ECU, my Ubuntu PC.

We know that there is room to improve. As resources are limited, what's your use case for these large messages, or are you testing the stack in general?

I am just testing and bootstrapping the titan TTCN test port by testing the boundaries.

[marckleinebudde]

I'm looking forward to a TTCN port for j1939.

[josemic]

Callingsend() with the remaining bytes, doesn't start a new message, the data is queued for the current running message transfer.

ssize_t send(int sockfd, const void *buf, size_t len, int flags);

When sending a second message, does the buffer buf need to stay the same like in the first message or the number of bytes sent with the first message have to be removed?

[olerem]

number of bytes sent with the first message have to be removed See: https://github.com/linux-can/can-utils/blob/master/jcat.c#L342

[josemic]

number of bytes sent with the first message have to be removed See: https://github.com/linux-can/can-utils/blob/master/jcat.c#L342

https://github.com/linux-can/can-utils/blob/master/j1939cat.c#L342 as file was renamed.

[marckleinebudde]

@josemic, is this issue solved for you now?

[josemic]

@marckleinebudde: For me it is solved. My TTCN code sends now multiple messages, if not all bytes have been transferred in first place. I find it quite unsatisfactory, to know that I am not able to determine the maximum message size. Isobus transfers for the Virtual Terminal binary blobs, which contain graphical objects. I do not know how large they may become today and in the future. Perhaps you have experience with that.

I think we should put at least something into the documentation, best would be, if we could provide a way how to determine and assure the maximum message size. As on my PC with multiple GB of RAM are just 4MB are possible, I am afraid of embedded hardware being unable e.g. after software update to transport the required maximum message size any longer. What do you think?

[marckleinebudde]

The TX size is not limited, as the kernel never holds the complete message in RAM. For RX the kernel first receives the complete message in the kernel and than forwards it into the user space. We cannot tell how big the max RX package is. I think the kernel reacts properly and sends the appropriate (E)TP error message. If needed the RX code path can be reworked in order to enable streaming of RX data.

[josemic]

I have sent a maximum size ETP message. I can confirm that message send returns a "Message too long" error message. I think we can close this issue.