read write frequency

[marvin-ad-martianum]

Im running a ros2 node reading and writing in parallel (optimized for minimal data) on four loops. However, the data transmission speed of the fastest loop is limited to 1000hz (also if the plc is running faster) and less if i am running everything at the speed i need (around 600hz for the fastest loop). Is there a way to speed up the beckhoff side? or change from tcp to udp? The general limitation seems to be around 800-1000 microseconds for one read or write process of a variable or list of variables.

[pbruenn]

You might be able to achieve a higher frequency with AdsSyncAddDeviceNotificationReqEx

When you set AdsNotificationAttrib.nCycleTime=0 and use ADSTRANS_SERVERCYCLE you should get the maximum amount of events.

A simple example for notifications is here: https://github.com/Beckhoff/ADS/blob/master/example/example.cpp#L30-L42

[marvin-ad-martianum]

Thank you for your response! I took your advice and had a try. Now please correct me or point me in the right direction.

Original Loop

This is how I had one of my loops: (some parameters are written, some are read. In the future, I will separate read and write to different arrays.)

// Initialize
AdsVariable<std::array<bool, machine_control_bool_len>> machine_control_bool{route, idx_group, idx_offset_BOOL};

// Pass to node as pointer

// -------- Start loop -----------------

std::array<bool, machine_control_bool_len> machine_control_b = static_cast<std::array<bool, machine_control_bool_len>>(machine_control_bool_);

// ---- Stay alive, ROS to Beckhoff
machine_control_b[0] = true;

// Do other stuff

// Write using the operator overloading of ADS template
machine_control_bool_ = machine_control_b; 

// ---- End loop ----------------------

New Approach with Callback

With the new approach you suggested, would I be callback-based? So Beckhoff would define the frequency through PLC timers, or how? I implemented this:

// Initialize

long nErr;
AmsAddr Addr;

// Set remote AmsNetId
Addr.netId = remoteNetId;

// Set remote AMS port (commonly 851 for the first PLC runtime system)
Addr.port = AMSPORT_R0_PLC_TC3;

// Index group and index offset
uint32_t indexGroup = idx_group;
uint32_t indexOffset = idx_offset_BOOL;

AdsNotificationAttrib NotificationAttrib;
NotificationAttrib.cbLength = sizeof(std::array<bool, machine_control_bool_len>);
NotificationAttrib.nTransMode = ADSTRANS_SERVERCYCLE;
NotificationAttrib.nMaxDelay = 1000; // 0.001 second
NotificationAttrib.nCycleTime = 0; // 100 ms

uint32_t hNotification;
uint32_t hUser = 0;

nErr = AdsSyncAddDeviceNotificationReqEx(AdsPort, &Addr, indexGroup, indexOffset, &NotificationAttrib, AdsNotificationCallback, hUser, &hNotification);

if (nErr) {
    std::cerr << "Error: AdsSyncAddDeviceNotificationReqEx failed with error code " << nErr << std::endl;
    return -1;
} else {                
    std::cerr << "Success on New callback method with: code " << nErr << std::endl;
}

void AdsNotificationCallback(const AmsAddr* pAddr, const AdsNotificationHeader* pNotification, uint32_t hUser) {
    // Suppress unused parameter warnings
    (void)pAddr;
    (void)hUser;

    const std::array<bool, machine_control_bool_len>& data = *reinterpret_cast<const std::array<bool, machine_control_bool_len>*>(pNotification + 1);
    std::cout << "First element: " << data[0] << std::endl;

    }

Questions and Considerations

• Frequency Control: With this approach, the frequency of notifications would be controlled by the PLC timer or the cycle time specified in NotificationAttrib.nCycleTime. Setting nCycleTime to 0 means the callback will be triggered as soon as possible after a change is detected?

• Data Handling in Callback: The callback is where youIwould save the data received from the PLC. If I need to write data back to the PLC, I can do so from within the callback or from another function, depending on my application's requirements?

• Writing Data: Writing data can still be done the same way as before. If I need to write data based on the received notification, I can do it inside the callback or schedule it for later execution in my main loop or another thread?

[pbruenn]

@marvin-ad-martianum

• Frequency Control: Yes, that is what I would expect. Your callback is called as fast as possible after a change but at least about once every nMaxDelay. However, I didn't actually tested it myself so I might be wrong.
• Data Handling in Callback: The callback is run in the context of the NotificationDispatcher of which we have one per AmsPort Notification. So I would rather consider a callback similar to an interrupt service routine and move the data as fast as possible to another thread for further processing. Trying to send data from that context will most likely deadlock/timeout your request similar as in https://github.com/Beckhoff/ADS/issues/173.
• Writing Data: should be done in a different thread context than the callback

EDIT: We seem to have one dispatcher thread per Notification, however to be able to receive the next one while processing another you should either process fast or relay the data to another thread

[marvin-ad-martianum]

Great. I have implemented this and it works amazing. I am in the nanosecond range for receiving data (much faster than expected). However, there is still one issue. Writing using the overload of the ads function such as before:

std::array<bool, machine_control_bool_len> machine_control_b = static_cast<std::array<bool, machine_control_bool_len>>(machine_controlbool);

// Do other stuff

// Write using the operator overloading of ADS template machine_controlbool = machine_control_b;

This operation delays the callbacks and creates gaps in the streamed data. Is there a more efficient way to writing data back to beckhoff? also with defining some sort of callback on the beckhoff side? This operation takes around 200 to 1200 microseconds which is extremely slow compared to the 10-30 nanoseconds i measure on the callback. Further the time it takes to write from cpp to ads i canot influence from the cpp side. beckhoff will let it rise to the large value of 1200 before it cycles down to 200. This operation seems to be blocking to some extent.

[pbruenn]

You can try to use a different "AmsPort" for sending than receiving. I don't know about the "TwinCAT receiving end" so I am sorry I cant help you with that.

Do you run TwinCAT and your ros2 node on the same CPU? Or is there a real network physical layer between them?

[marvin-ad-martianum]

Yea. okay. Do you know that using another port is nonblocking?

I was thinking of adding another remoteNetId but the port would be more convenient i guess.

    AmsAddr Addr;
    // Set remote AmsNetId
    Addr.netId = remoteNetId;
    // Set remote AMS port (commonly 851 for the first PLC runtime system)
    Addr.port = AMSPORT_R0_PLC_TC3;

I physically divide the system. I have the beckhoff plc on a their classical windows and a 32 core linux server to do the heavy lifting. The software is multi-threaded and the bottleneck the ads tcp i think (also udp seems impossible with the current state of beckhoff). The callback seems almost unreasonably fast while writing is slow at 1ms.

[pbruenn]

Okay with (virtually) two separate hosts maybe EtherCAT Automation Protocol (EAP) is a better solution. Did you considere that: https://infosys.beckhoff.com/content/1033/eap/1521519371.html?id=795366871337330973

[marvin-ad-martianum]

Yes, Two considerations, one we are planning to switch beckhoff to linux for shipping later, which they claim to "release in the future". Whatever that means. But its important to have this option to move to one "pc" unit.

I have seen the advantage of Ethercat but no route that would allow me to use the current hardware C6030-0070 without a lot of extra work and without the guarantee it will greatly improve the performance. If there is an efficient way we will do it, if there is not i will live with the limitation. I saw a few things but do you know some reliable library?

Is there a way in this library to limit or define the write time? switch to udp? or i guess changing the port could solve the issue? Given i am 90% at where i want to be i am inclined to stay with this solution here.

[pbruenn]

Yeah, I totally understand your reasoning.
I looked into the code again and the problem is we only have a single TCP connection. I don't know how good both sides can handle send/recv in parallel. So that might be one of the limits. UDP would be the obvious next try. But the problem is the TwinCAT cannot receive ADS on UDP (As far as I know).

I have zero experience with EAP myself, so I can't tell you how much effort it is to port your current solution to EAP. And I can't tell about any libraries either. I simply don't know. What I know is the EtherCAT Technology Group has some excellent support guys. Not sure if you have to be ETG member (free of charge) to ask them for support. If you get EAP working I think you should make it work on localhost too, with virtual network interfaces or UDP.

Another hack you can try is socat. You can try to run socat as a UDP tunnel between Linux and Windows and have TCP only on localhost. I can imagine this might speed up your latency issue, but getting the ADS setting correct for this might be pretty hard. You might need a fake IP address on the windows side to force TwinCAT really send tcp out to your socat instance on localhost.

[marvin-ad-martianum]

Okay. Thank you for the update and the details. I will live with the limitation for now and maybe update to TwinCat someday.

What I hope to understand at some point is why i can receive information within nanoseconds while writing takes microseconds and this somewhat randomly between 200 to 1200 us. I will try and contact them directly. We do get support.

[pbruenn]

I don't believe you really get updates within nanoseconds. What I believe is you measure the time between two calls of your callback.The difference is the tcp stack can buffer incoming packets for you so while your initial receive is delayed a bit. future receives are already on the host when you process the first one. So your callback gets called much rapidly. For writing it is much harder because when you "send()" you have to wait for the ACK response.

[marvin-ad-martianum]

Yes, that makes sense, this is what i expected but didn't understand why. Thank you for the clarification.

So its basically two different mechanisms, one is a callback i am waiting for, which executes at the set loop time or max frequency of the beckhoff system part which is writing the variable. The other one is a write from my linux using a ack which seems to be partially blocking or delaying the callbacks during this time. But the time it takes to write to a beckhoff variable is bigger than the data-gap in the callback data receiving from it.

Its good enough for now.