kuka mxautomation_hw_interface support

[yijiangh]

Refer to the discussion in swri-ros-pkg-dev google group discussion.

I'm proposing the development of support for kuka mxautomation[3, 4], a new interface package provided by kuka. It seems like kuka has stopped selling rsi package and we cannot find it on their website anymore.

The attached pictures are some of the quick glimpse on mxautomation's feature.

Happy to hear opinions and suggestions.

Best, Yijiang

Reference: [1] Munz, Heinrich, Johannes Braumann, and Sigrid Brell-Cokcan. "Direct Robot Control with mxAutomation: A New Approach to Simple Software Integration of Robots in Production Machinery, Automation Systems, and New Parametric Environments." Robotic Fabrication in Architecture, Art and Design 2016. Springer International Publishing, 2016. 440-447. [2] Braumann, Johannes, and Sigrid Brell-Cokcan. "Adaptive Robot Control." Proceedings of the 33rd eCAADe Conference, Vienna, Austria. 2015. [3] https://www.kuka.com/en-us/products/robotics-systems/software/hub-technologies/kuka,-d-,plc-mxautomation [4] http://store.codesys.com/mxautomation.html?___store=en&___from_store=default

Figure 1 - mxAutomation control concept. Software components provided by KUKA are marked orange. (Picture courtesy from [1])

Figure 2 - Location of command logic: Entirely on the robot controller for regular control data files (SRC), entirely on the external controller for real-time controllers (RSI), distributed command logic (NEW) (Picture courtesy from [2])

[gavanderhoorn]

Hi @yijiangh: thanks for starting discussion on an mxAutomation driver/interface.

As I also wrote in my reply to your post to the ROS-Industrial mailing list, I'd be interested in getting such a package included in this repository.

Unfortunately though I don't have access to any hardware that supports this, so any development would have to be contributed by community members such as you. I can support you with my experience with the ROS-side of all of this though.

Do you already have more concrete plans for this, or are you still just gathering input?

---

Btw:

It seems like kuka has stopped selling rsi package and we cannot find it on their website anymore.

I'm not quite sure this is true. As also stated by @BrettHemes in the ROS-Industrial mailing list post you linked to, as far as I'm aware, RSI should still be available, but it may not be too present in KUKA's marketing material now with the availability of mxAutomation.

I've just sent an email to a contact at KUKA to see whether we can get a clear answer on the status of RSI.

[gavanderhoorn]

I can find very little public documentation on mxAutomation btw, so if you have anything you can share that would make discussion a bit more meaningful.

[yijiangh]

Hi @gavanderhoorn , thanks for your reply.

Do you already have more concrete plans for this, or are you still just gathering input?

Currently no, actually my lab are negotiating with kuka to get MxA. Also I'm still open and gathering inputs.

I can find very little public documentation on mxAutomation btw, so if you have anything you can share that would make discussion a bit more meaningful.

Sure, here're some papers [1, 2, 3] related to MxA-related experiments in creative industry and documentation [4] for KUKA.PLC mxAutomation 2.1.

[1] Brell-Cokcan, S and Braumann, J 2015, ‘Towards Adaptive Robot Control Strategies’, Proceedings of the 35th ACADIA Conference, Cincinnati, USA [pdf]

[2] Braumann, J and Brell-Cokcan, S 2015, ‘Adaptive Robot Control’, Proceedings of the 33rd eCAADe Conference, Vienna, Austria [pdf]

[3] Munz, Heinrich, Johannes Braumann, and Sigrid Brell-Cokcan. "Direct Robot Control with mxAutomation: A New Approach to Simple Software Integration of Robots in Production Machinery, Automation Systems, and New Parametric Environments." Robotic Fabrication in Architecture, Art and Design 2016. Springer International Publishing, 2016. 440-447. [pdf]

[4] CODESYS Library for KUKA.PLC mxAutomation 2.1 [pdf]

[gavanderhoorn]

I was looking more for public documentation on the protocol or the way PLC vendors are supposed to integrate with mxAutomation supporting motion controllers. The papers you link are very high-level descriptions of what should be / is possible with the interface, and the CODESYS library doc seems to be just that, a reference for using the CODESYS mxAutomation library.

I can't really find any OSS implementations of mxAutomation compatible libraries (or any implementations really, apart from automation vendors), so that would seem to be a first major issue to tackle.

Documentation on the protocol - if any - would certainly help with that.

The KUKA|prc devs appear to have implemented something, but I can't see what they are doing as the source does not seem available.

[gavanderhoorn]

It seems like kuka has stopped selling rsi package and we cannot find it on their website anymore.

@yijiangh: I've just had it confirmed by my KUKA contacts that RSI is still available and here to stay. mxAutomation is not a replacement and does not sit in the same niche as RSI.

The fact that you may not be able to find it on the website is a content management issue, apparently.

[gavanderhoorn]

@yijiangh: have you had any chance to look into this? mxAutomation support is still of interest, but without access to the required hardware & software is going to depend on a community contribution.

[yijiangh]

@gavanderhoorn : Sorry for not updating for so long, my lab is still working on acquiring it from KUKA. will update as soon as we get it!

[BrettHemes]

@gavanderhoorn @yijiangh I have started talking with our Kuka reps about this and learned that the newest version of mxAutomation provides C# and C++ API's for PC interfacing, however, they are currently distributed only as Windows binaries.... I will continue to look into this as I see it as a potential happy middle-ground between the EKI and RSI approaches (i.e., should be higher-performance than the EKI while also relaxing the (hard) real-time constraints of the RSI approach due to buffering of incoming trajectory points).

Reading some of the KUKA|prc stuff seemed to imply that the source is available from Kuka but requires a confidentially agreement... not sure how that would work out in the scope of ROS

[edit]: RSI, not SPI...

[gavanderhoorn]

middle-ground between the EKI and SPI approaches

RSI you mean?

Reading some of the KUKA|prc stuff seemed to imply that the source is available from Kuka but requires a confidentially agreement... not sure how that would work out in the scope of ROS

well .. we'd either need some kind of protocol spec, or there would need to be a library available for Linux that does the protocol handling.

If neither of those are available, reverse engineering the protocol spec could be an option, but only as a last resort.

How do PLC vendors interface with this? Is that all by incorporating some binary blob from KUKA? Or are they covered by the NDA?

Perhaps we could try and get KUKA interested in this? Would your KUKA rep be in a position to facilitate anything?

[gavanderhoorn]

@yijiangh: now that @BrettHemes has submitted a candidate EKI driver, I'm curious to know whether you've had any chance to take a look at mxAutomation in the meantime.

@BrettHemes: you mention that you wanted to look into the libraries provided by KUKA. Have you had a chance to do that? We may be able to decompile the C# libs - provided they don't just wrap the C++ ones.

[yijiangh]

@gavanderhoorn Thanks for the follow-up. Due to funding issues, I think my lab won't be able to acquire MxAutomation before September, 2018. So I am sorry to say that I won't be able to offer any real hardware-based experiments to contribute to our efforts here at this moment.

However, even if I don't have the hardware to test it, I did manage to acquire the C# source code (with MxAutomation 2.1.3 dll library) for a windows-based UDP server to connect to the MxAutomation client running in kuka controller box. I was told that this approach has been tested on real hardware and it worked like our familiar joint state publisher in ROS.

I have to keep the source in private repo due to confidential issues but I've invited @gavanderhoorn and @BrettHemes to the repo. Maybe we can try on decompiling the dll library to see if there is any direction that we can work something out.

[gavanderhoorn]

From what I've seen so far there appears to be very little actual native (ie: Win32) code in there. As we have .net core on Linux these days, it might be worthwhile to try and run the application in your repository directly. If we can do that, and it works with an actual robot, we could potentially create a driver in .net, with the KUKA assembly doing all the heavy lifting.

That would also mean that we can just distribute the driver sans KUKA proprietary mxAutomation lib, which would then have to be procured by users themselves. This would be similar to how many KUKA FRI software components are distributed, and avoids licensing and various other legal issues completely.

I'd be very interested in getting something like that to work, however I don't have any hardware to test anything.

@BrettHemes: do you have access to any mxAutomation hardware?

[gavanderhoorn]

Looking through the example application included, it looks like the actual mxAutomation functionality is completely encapsulated by the provided assembly.

Data coming from the controller is read from a UDP socket and the buffer is then passed to the assembly for parsing. Setting new axis goal positions, toggling IOs, etc is all done on the same assembly objects, and those actions are then serialised into an application provided byte buffer which is sent to the controller again over the same socket.

Such a design makes sense, as it completely decouples the transport from the proprietary mxAutomation IP and protocol (de)serialisers. It also makes application design relatively straightforward, as there is only a very simple interface between an application and the mxAutomation assembly. Control over data going in and out of the application is also completely left to the application, with the assembly objects control flow being driven by that same application.

Conceptually this looks like the following:

while True:
  buf = read_socket(..)
  mxa_in.data = buf
  mxa_in.update()

  # insert application logic here, sets properties of mxa_out based on state of mxa_in

  out_buf = mxa_out.update()
  write_socket(out_buf, ..)

  sleep(..)

Being a native .net DLL might actually make the idea I posted in my previous comment possible: it doesn't look like the assembly uses anything apart from standard .net functionality, which should be supported by the .net core runtime.

[gavanderhoorn]

Linking this here for future reference: mono/Embeddinator-4000.

[gavanderhoorn]

I've done some minimal tests (got nerd-sniped) with .net core 2 and a very minimal c# prog (basically a stripped version of the MxAutomation_Example application) and so far things compile and run under Ubuntu 16.04 amd64 (with 'run' I mean: no exceptions, no complaints about loading assemblies from disk, etc, not really communicating with anything here).

As I have no robot to test with that is all I can do.

If anyone is interested in continuing this post a comment here and we'll see how we can collaborate.

If you are in Europe and are interested in this, this could be a good ROSIN FTP.

[BrettHemes]

XKCD ftw :+1:

@BrettHemes: do you have access to any mxAutomation hardware?

If by hardware you mean PLCs and the like, not in my possession currently but they are plentiful here and I could get one rather easily. As for the mxAutomation license, none of my lab robots have this installed. That is not to say we couldn't get it if it makes sense...

[BrettHemes]

but I've invited @gavanderhoorn and @BrettHemes to the repo.

@yijiangh Thanks, but I seem to have missed this somehow and can't find the invite?

[yijiangh]

@BrettHemes Just sent invitation again.

[gavanderhoorn]

@BrettHemes wrote:

If by hardware you mean PLCs and the like, not in my possession currently but they are plentiful here and I could get one rather easily. As for the mxAutomation license, none of my lab robots have this installed. That is not to say we couldn't get it if it makes sense...

No, I meant mxAutomation license on any KR C4.

Without hw that responds to whatever the C# application sends, we're not going to get very far.

[gavanderhoorn]

Btw: we may have gone over this before, but @yijiangh: where would mxAutomation be positioned relative to EKI? To me it seems like it's relatively similar.

[yijiangh]

@gavanderhoorn I have no experience wth EKI but based on the discussion in #77 , I feel like EKI is a program upload service to replace the default USB-based copy-and-paste appoach to load KRL file into KUKA controller.

Like you've said in #77:

... it allows us to close the (feedback) loop on a low-level joint space or Cartesian position controller over a high-bandwidth, low-latency interface. That is not something that most other simple_message based drivers do (although it is possible).

The main point of adding MxAutomation for me is its potential of enabling these sensor-based online control, as a complement to existing RSI approach.

[gavanderhoorn]

I feel like EKI is a program upload service to replace the default USB-based copy-and-paste appoach to load KRL file into KUKA controller.

I don't believe that is completely accurate: see #110 contributed by @BrettHemes.

From what I've read, mxAutomation is like EKI, but implemented in such a way that it's easy to integrate with the control infrastructures of more 'traditional' PLCs based on function-blocks and ladder programs. This would allow PLCs to completely control the motion of a robot, completely removing the need to program in KRL.

[yijiangh]

@gavanderhoorn Oops. Sorry for my ignorance... Then based on my understanding, I agree with @BrettHemes 's comment here:

I will continue to look into this as I see it as a potential happy middle-ground between the EKI and RSI approaches (i.e., should be higher-performance than the EKI while also relaxing the (hard) real-time constraints of the RSI approach due to buffering of incoming trajectory points).

[gavanderhoorn]

No need to apologise.

I'm not sure whether mxA ~= EKI, that is just my interpretation of things.

Note that I'm also not for or against adding mxAutomation support if/when we have EKI support: some users may have EKI but not mxAutomation or the other way around (given my current understanding I would find that a likely situation). Adding support for both would just increase the utility of the pkgs in this repository.

[BrettHemes]

Just to chime in, I now think that the mxA approach is the most limited of all three mentioned above (but see bold comment below). I have yet to really dig in but from looking at the documentation it looks to me like the intended behavior is to invoke a subset of KRL functionality (e.g., no splines) through a block-like interface from within a PLC. These blocks resolve in the manner that @gavanderhoorn figured out above. The list of available motion commands (via the mxAutomation Logix 1.0 - KSS 8.2 documentation) are:

• KRC_MoveLinearAbsolute
• KRC_MoveLinearRelative
• KRC_MoveDirectAbsolute
• KRC_MoveDirectRelative
• KRC_MoveAxisAbsolute
• KRC_MoveCircAbsolute
• KRC_MoveCircRelative
• KRC_JogLinearRelative
• KRC_JogToolRelative

All said, I don't believe a direct comparison is really possible as the EKI package provides general Ethernet communication capability that can then become anything within the realm of KRL capability (e.g., download interface, streaming interface as in #110, mxAutomation-like interface, etc.).

Note that I'm also not for or against adding mxAutomation support if/when we have EKI support: some users may have EKI but not mxAutomation or the other way around (given my current understanding I would find that a likely situation). Adding support for both would just increase the utility of the pkgs in this repository.

This.

[BrettHemes]

Also, in general, if you are using PLC's are you still using ROS?

[gavanderhoorn]

Off-topic: but there are PLCs that have/will get ROS support.

[BrettHemes]

Ahhh, so fast! @gavanderhoorn can read minds :smile:

[gavanderhoorn]

Note that the listing in the Logix manual does not appear to cover everything that is part of the software @yijiangh shared. Those are the higher-level blocks, but it could be that more 'advanced' control can be implemented using the rest of the infra.

[BrettHemes]

I assumed such... thanks for the heads up. I will continue digging.

[gavanderhoorn]

I just learned that we do actually have a KRC4 with mxAutomation here at the university, so I may be able to test some things in the near future. As I have a lot of other things as well, it may be some time before I can do anything, so my offer of assisting interested parties stands.

[gavanderhoorn]

@yijiangh: any updates here?

[yijiangh]

@gavanderhoorn Trying to push my lab for a purchase of the MxAutomation package during the summer! I'll come back and report as soon as we have it.

[gavanderhoorn]

Trying to push my lab for a purchase of the MxAutomation package during the summer!

As long as it's not on our behalf :)

[yijiangh]

@gavanderhoorn oh definitely not... There are some requests from our student on using this too. So in this case, it's a purchase qualified for both research and educational purposes :)

[yijiangh]

@gavanderhoorn Sorry I am just being told that my lab does not have a plan to acquire MxAutomation in short term. Thus, I have to say that the development in MxAutomation support can't get very far on our side without the hardware. 😢

[gavanderhoorn]

That is unfortunate.

We'll keep this open for a little while longer (so as to keep the issue visible), but we'll close it if there are no further updates.

@BrettHemes: any updates from your side?

[daneisinger]

I would be interested in helping with this in some capacity. We have a KR60-3 with KRC4 and an mxAutomation license (also just purchased an RSI license EDIT: and EKI license). I have yet to do any substantial testing of mxA, and I am new to ROS, but I am eager to immerse myself as this may be foundational to our research going forward. It would be great to be able to leverage ROS with our KUKA platform.

[gavanderhoorn]

That would be great @daneisinger.

For a driver not a lot of ROS knowledge/experience would be needed, although for testing it you'd probably need something of a working setup.

The first task would probably be to get a feeling for what mxAutomation can and cannot do. Typicaly ROS robot drivers expose a FollowJointTrajectory action interface, which operates in joint space. I see some references to non-Cartesian motions in the listing @BrettHemes provided, so that should be possible.

The other functionality might be worth exposing as well, but we can look into that at a later time.

[daneisinger]

Yes, non-Cartesian motions are definitely possible through the KRC_MoveAxisAbsolute function block. There are also function blocks for reading current axis positions (and other info) and doing I/O.

Note that the listing in the Logix manual does not appear to cover everything that is part of the software @yijiangh shared. Those are the higher-level blocks, but it could be that more 'advanced' control can be implemented using the rest of the infra.

I believe I have the same software (the mxAutomation_Example UDP server) and I'm curious--what functionality did you see in the software that did not appear to be covered in the manual?

[gavanderhoorn]

@daneisinger wrote:

@gavanderhoorn

Note that the listing in the Logix manual does not appear to cover everything that is part of the software @yijiangh shared. Those are the higher-level blocks, but it could be that more 'advanced' control can be implemented using the rest of the infra.

I believe I have the same software (the mxAutomation_Example UDP server) and I'm curious--what functionality did you see in the software that did not appear to be covered in the manual?

This was some time ago, and I don't have my notes with me. I was looking at the provided assemblies at the time using a CLR disassembler and I believe I found references to (internal) things that did not appear to be documented.

Whether those are things we want to use, not sure.

[farshbafdoustar]

Any news on this topic?

I'm interested in controling KUKA with mxAutomation over PROFINET. Based on this document there is already an implemented PROFINET interpreter for mxAutomation . What I have found in KUKA documentations is information about the PLC-OPEN block diagrams ready to use with Siemens S7. And based on the above mentioned document, it is possible to control the Kuka robot with S7 over Telegram 2032 (programming visually in TIA portal).

As I have already integrated PROFINET over Siemens CP1626 inside my ROS platform, I'm interested to control the arm also over PROFINET as an IO device. My current blocking point is some missing information about the mapping between Telegram 2032 Bytes and I/O data used in the PLC-OPEN block diagrams.

Any Idea?

@daneisinger : Are you using UDP for mxAutomation communication? Any advancement on the implementation? I'm also interested to contribute.

Regards, Behnood

[gavanderhoorn]

I'm not aware of any progress on this front.