Question: Setup Java Driver Software

[tdinesh]

I am having issue with the Kuka LBR iiwa Java Setup - "Setup Java Driver Software". The wiki is bit not clear to me.

Copy The folder contents ${GRL_DIR}/src/java/grl/ into the directory where your Sunrise Project is located. Does this mean copy the (.settings, src, Third PartyLibraries, .classpath, .project) into my project directory? Or copy the "grl" folder as a whole under project_dir.

Could you please clarify this. Detailed folder structure/names would be quite helpful. Thanks!

[ahundt]

I think I usually copy the whole grl folder, sometimes I have to add paths to sunrise or Chang the imports a bit. I would just commit a whole project but sunrise syncs fit repositories it creates hidden behind the scenes.

On Wed, Feb 1, 2017 at 3:28 PM tdinesh notifications@github.com wrote:

I am having issue with the Kuka LBR iiwa Java Setup - "Setup Java Driver Software". The wiki is bit not clear to me.

Copy The folder contents ${GRL_DIR}/src/java/grl/ into the directory where your Sunrise Project is located. Does this mean copy the (.settings, src, Third PartyLibraries, .classpath, .project) into my project directory? Or copy the "grl" folder as a whole under project_dir.

Could you please clarify this. Detailed folder structure/names would be quite helpful. Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ahundt/grl/issues/119, or mute the thread https://github.com/notifications/unsubscribe-auth/AADZwDNQIVvt31yKXYCh7csP-HKCP1Ljks5rYOrmgaJpZM4L0WLY .

-- Cheers! Andrew Hundt

Ph.D. Student Johns Hopkins University

[ahundt]

Did that help or do you need more info like a screenshot of my own setup?

[tdinesh]

Screenshots would be useful. I am getting warnings/exclamations on the project build. Thanks!

[ahundt]

Yeah there are usually a few that need to be fixed based on changes in path or what software kuka makes available. Sunrise is eclipse under the hood so sometimes you need to add extra folders to the classpath, and you'll need the JARs for flatbuffers. Do you have the connectivity suite?

Here is how I've got my folder layout set up:

[ahundt]

Some of those folders are internal stuff for me or legacy which won't matter to you. For example JeroMQ should no longer be needed, and you should not have the "de" folder nor the "com" folder.

Sorry the Java side isn't in better shape I'm only moderately experienced in managing Java projects. Furthermore, Kuka makes weird nonstandard things happen like their annoying hidden automatic git repository, plus they change their library design and available executables depending on your hardware... Really poorly done by them.

[tdinesh]

I have Sunrise Workbench - 1.11.0.7 with the connectivity suite. For flatbuffers do you just include the "flatbuffers/java/com/google/flatbuffers" folder in the project? Attached is my current workspace screenshot

[tdinesh]

Managed to fix the errors and transfer the application to the controller. Now running the GRL_Driver gives following error. @ahundt any thoughts?

[ahundt]

Oh yeah there is an XML file where all that sort of data should be loaded from. In the kuka documentation that will probably be referred to as "ProcessData"

They are the xml files right in this folder: https://github.com/ahundt/grl/tree/master/src/java/grl/src

And I have code in this file that loads it: https://github.com/ahundt/grl/blob/master/src/java/grl/src/grl/ProcessDataManager.java

[ahundt]

This is the one that specifically contains Laptop_IP: https://github.com/ahundt/grl/blob/master/src/java/grl/src/RoboticsAPI.data.xml

So you need to make sure it is in the right spot in your sunrise project that it gets loaded up (and that the values are set properly for your system).

[ahundt]

Also here is the folder with the flatbuffer jar I use. I haven't quite decided the best way to put the code in so I never committed it. I'm sure maven or something would be the way to do it... Just not sure so if you know more about Java package managment and can provide advice it would be appreciated!

FlatBuffer-1.3.zip

I should probably also update to 1.5...

[ahundt]

Also if you're planning to use FRI directly, it can command the robot to move but there is one small bug that needs to be fixed to make sure that when you give an absolute joint angle it goes to that absolute joint angle with all zero angles being straight up. FRI is designed strangely to be interpolated motions over some other command so figuring out how to make it respond the way you would normally expect to control joints has been a bit of an adventure...

Are you planning on torque level control as well?

[ahundt]

In other words, don't just run FRI without watching very carefully. Be ready to hit the red button to prevent the arm from crashing if it doesn't go where you expect!

It is very, very close to working properly though, here is what I'm doing to finalize the last of these details in case you'd like to replicate:

KukaFRIClientDataDriverTest

I suggest you start with: https://github.com/ahundt/grl/blob/master/test/KukaFRIClientDataDriverTest.cpp

This should be compiled in the build/test/bin folder or whatever your cmake build folder is called. This should move the arm to a starting position and rotate the last joint only. This is just a test app for the API.

FRIHoldsPosition_Command KukaFRIClientDataDriverTest should be run alongside FRIHoldsPosition_Command. This is just a test app for the API.

https://github.com/ahundt/grl/blob/master/src/java/grl/src/grl/driver/FRIHoldsPosition_Command.java

TODOs

1. The one TODO for position based FRI control to work is mentioned at this line: https://github.com/ahundt/grl/blob/master/test/KukaFRIClientDataDriverTest.cpp#L157
   • The same sort of fix and ensuring the joint angles behave correctly will need to be fixed similarly to this todo in the actual FRI driver class.
2. Then for torque there may be one or two spots where the data/commands need to be copied into and out of the protocol buffer and into a more accessible format.

Running FRI with full driver

Once that todo line referenced above is fixed and the class works correctly, it should be quite easy to use either the ROS driver, V-REP driver, or directly use the grl class to control the robot in JAVA (really smartservo) or FRI mode by just changing the configuration string when initializing the KukaDriver object like in the following default configuration code: https://github.com/ahundt/grl/blob/master/include/grl/kuka/KukaDriver.hpp#L84

[ahundt]

Also note: the JAVA mode (SmartServo) works very reliably. We've been using that mode for two years now, which is why FRI isn't 100% yet. SmartServo is good enough for many use cases + demos, but I'm just getting to the point where I need that higher level of precision myself.

[tdinesh]

Thanks a lot for the detailed explanations.

I am first trying to get the JAVA mode running. Still having issues with that. The GRL_Driver now works on the KUKA controller. But the grl_kuka_ros_driver gets a segfault.

/opt/grl-4.1.0/bin/grl_kuka_ros_driver [ INFO] [1486402463.842206456]: done creating subscribers Segmentation fault (core dumped)

Did you manually set the Robot_KONI_FRI_IP on the robot controller in the windows embeded? I have reverted the KONI changes from https://github.com/SalvoVirga/iiwa_stack/wiki/cabinet_setup. Wondering if something is wrong in that front. Thoughts?

Yes, I am planning to use FRI and torque control. Will switch to testing it once the JAVA mode works.

[tdinesh]

Also, I dont have much experience in Java package managment. So not sure what is the best way to included external flatbuffer dependency.

[ahundt]

On Mon, Feb 6, 2017 at 1:45 PM tdinesh notifications@github.com wrote:

Also, I dont have much experience in Java package managment. So not sure what is the best way to included external flatbuffer dependency.

I think you can just put it in the folder right click and add it to the path or something. Otherwise just google how to add a jar file to eclipse

—

You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/ahundt/grl/issues/119#issuecomment-277774178, or mute the thread https://github.com/notifications/unsubscribe-auth/AADZwKewTddXGN7fqoEVsoaCPejiOY2Yks5rZ2pbgaJpZM4L0WLY .

--

Cheers! Andrew Hundt

[tdinesh]

Oh, I know how to include flatbuffer in Sunrise Workbench. I was wondering if there is a better way as you had asked earlier. Still struggling with the segfault

[ahundt]

For the segfault you will need to run in a debugger and get me a stack trace. Then I can probably help.

On Mon, Feb 6, 2017 at 2:30 PM tdinesh notifications@github.com wrote:

Oh, I know how to include flatbuffer in Sunrise Workbench. I was wondering if there is a better way as you had asked earlier. Still struggling with the segfault

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/ahundt/grl/issues/119#issuecomment-277786647, or mute the thread https://github.com/notifications/unsubscribe-auth/AADZwHU5XvUt5IcPSFJbGM01yQ1rQwnpks5rZ3TFgaJpZM4L0WLY .

--

Cheers! Andrew Hundt

[tdinesh]

Following is a backtrace with gdb. Thanks!

` (gdb) trace Tracepoint 1 at 0x518a20 (gdb) backtrace

0 0x0000000000518a20 in spdlog::logger::should_log(spdlog::level::level_enum) const ()

1 0x000000000053683e in spdlog::details::line_logger spdlog::logger::_log_if_enabled<std::cxx11::basic_string<char, std::char_traits, std::allocator >, char [2], std::cxx11::basic_string<char, std::char_traits, std::allocator >, char [5], std::__cxx11::basic_string<char, std::char_traits, std::allocator > >(spdlog::level::level_enum, char const*, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, char const (&) [2], std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, char const (&) [5], std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) ()

2 0x000000000052a7c0 in spdlog::details::line_logger spdlog::logger::info<std::cxx11::basic_string<char, std::char_traits, std::allocator >, char [2], std::cxx11::basic_string<char, std::char_traits, std::allocator >, char [5], std::cxx11::basic_string<char, std::char_traits, std::allocator > >(char const*, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, char const (&) [2], std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, char const (&) [5], std::cxx11::basic_string<char, std::char_traits, std::allocator > const&) ()

3 0x000000000051becf in grl::robot::arm::KukaJAVAdriver::construct(std::tuple<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > >) ()

4 0x000000000051bcfd in grl::robot::arm::KukaJAVAdriver::construct() ()

5 0x000000000051eee3 in grl::robot::arm::KukaDriver::construct(std::tuple<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > >) ()

6 0x000000000051e52f in grl::robot::arm::KukaDriver::construct() ()

7 0x0000000000521728 in grl::ros::KukaLBRiiwaROSPlugin::construct(std::tuple<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator > >) ()

8 0x0000000000520d6f in grl::ros::KukaLBRiiwaROSPlugin::construct() ()

9 0x0000000000504eed in main ()`

[tdinesh]

Compiled the driver in Debug Build. Following is the backtrace.

` Thread 1 "grl_kuka_ros_dr" received signal SIGSEGV, Segmentation fault. 0x0000000000518a20 in std::atomic_base::load (__m=std::memory_order_relaxed, this=0x50) at /usr/include/c++/5/bits/atomic_base.h:396 396 return atomic_load_n(&_M_i, __m); (gdb) backtrace

0 0x0000000000518a20 in std::__atomic_base::load (__m=std::memory_order_relaxed, this=0x50) at /usr/include/c++/5/bits/atomic_base.h:396

1 spdlog::logger::should_log (this=0x0, msg_level=spdlog::level::info) at /usr/include/spdlog/details/logger_impl.h:295

2 0x000000000053683e in spdlog::logger::_log_if_enabled<std::cxx11::basic_string<char, std::char_traits, std::allocator >, char [2], std::cxx11::basic_string<char, std::char_traits, std::allocator >, char [5], std::__cxx11::basic_string<char, std::char_traits, std::allocator > > (this=0x0, lvl=spdlog::level::info, fmt=0x573028 "KukaLBRiiwaRosPlugin: Connecting UDP Socket from ")

at /usr/include/spdlog/details/logger_impl.h:79

3 0x000000000052a7c0 in spdlog::logger::info<std::cxx11::basic_string<char, std::char_traits, std::allocator >, char [2], std::cxx11::basic_string<char, std::char_traits, std::allocator >, char [5], std::__cxx11::basic_string<char, std::char_traits, std::allocator > > (this=0x0, fmt=0x573028 "KukaLBRiiwaRosPlugin: Connecting UDP Socket from ") at /usr/include/spdlog/details/logger_impl.h:117

4 0x000000000051becf in grl::robot::arm::KukaJAVAdriver::construct (this=0x7ddfe0, params=std::tuple containing = {...}) at /home/dinesh/ws_gsk/grl/include/grl/kuka/KukaJAVAdriver.hpp:160

5 0x000000000051bcfd in grl::robot::arm::KukaJAVAdriver::construct (this=0x7ddfe0) at /home/dinesh/ws_gsk/grl/include/grl/kuka/KukaJAVAdriver.hpp:148

6 0x000000000051eee3 in grl::robot::arm::KukaDriver::construct (this=0x7dda80, params=std::tuple containing = {...}) at /home/dinesh/ws_gsk/grl/include/grl/kuka/KukaDriver.hpp:134

7 0x000000000051e52f in grl::robot::arm::KukaDriver::construct (this=0x7dda80) at /home/dinesh/ws_gsk/grl/include/grl/kuka/KukaDriver.hpp:94

8 0x0000000000521728 in grl::ros::KukaLBRiiwaROSPlugin::construct (this=0x7d4270, params=std::tuple containing = {...}) at /home/dinesh/ws_gsk/grl/include/grl/ros/KukaLBRiiwaROSPlugin.hpp:195

9 0x0000000000520d6f in grl::ros::KukaLBRiiwaROSPlugin::construct (this=0x7d4270) at /home/dinesh/ws_gsk/grl/include/grl/ros/KukaLBRiiwaROSPlugin.hpp:144

10 0x0000000000504eed in main (argc=1, argv=0x7fffffffd2e8) at /home/dinesh/ws_gsk/grl/src/ros/grl_kuka_ros_driver.cpp:16`

[tdinesh]

KukaFRIClientDataTest seems to spit out the joint positions. [console] [info] position: [1.2164e-05, 0.436339, 0.000121999, 1.57083, -4.45813e-05, 6.59135e-06, -7.15458e-05] us: 0 connectionQuality: 3 operationMode: 2 sessionState: 2 driveState: 0 ipoJointPosition: [] ipoJointPositionOffsets: []

[ahundt]

In classes/meetings at the moment but should be able to look in detail tonight sorry for the delay.

On Mon, Feb 6, 2017 at 4:02 PM tdinesh notifications@github.com wrote:

KukaFRIClientDataTest seems to spit out the joint positions. [console] [info] position: [1.2164e-05, 0.436339, 0.000121999, 1.57083, -4.45813e-05, 6.59135e-06, -7.15458e-05] us: 0 connectionQuality: 3 operationMode: 2 sessionState: 2 driveState: 0 ipoJointPosition: [] ipoJointPositionOffsets: []

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/ahundt/grl/issues/119#issuecomment-277812353, or mute the thread https://github.com/notifications/unsubscribe-auth/AADZwLHeASR2gBW8vR33Q-hsr8kQ35qhks5rZ4p3gaJpZM4L0WLY .

--

Cheers! Andrew Hundt

[ahundt]

for the FRI test, make sure you have no other software running. The connection quality for commanding motion is 4, higher is better.

For the other one, try commenting the spdlog printouts at /home/dinesh/ws_gsk/grl/include/grl/kuka/KukaJAVAdriver.hpp:160 and wherever else it may be crashing. Since they are just printouts it should still run... there just won't be the debug details.

[ahundt]

Also that's just the quick and dirty way to fix it, usage of spdlog is found here: https://github.com/gabime/spdlog

It may be that the logger hasn't been initialized properly or its trying to initialize twice when it should be acquiring an existing instance. Usually pretty easy to fix but I'm not sure I've seen the exact error.

Here is an example of initializing it: https://github.com/ahundt/grl/blob/master/src/v_repExtGrlInverseKinematics/v_repExtGrlInverseKinematics.cpp#L131

Yes I think that's the most likely problem especially in the ros plugin which hasn't been run for a few months so it should be an easy fix.

[tdinesh]

Thanks! that fixed the segfault. Now having following issue Error sending packet to KUKA iiwa: ret = -1, len = 144 [ INFO] [1486491078.220977800]: KukaLBRiiwaROSPlugin in unsupported mode! Looks like I have to publish /interaction_mode? Can't the driver just be set to read without choosing mode?

[tdinesh]

Following is the error on the SmartPad. NULL ArmControlState message, main UDP FlatBuffer message is arriving but doesn't contain any data/commands!

[tdinesh]

I tried publishing /interaction_mode to MoveArmJointServo. It looks like the driver commands joint positions continuously. Is my interpretation correct? Why not latch onto last position (or last read position) until new position is commanded? I am looking from moveit perspective, it only publishes joint commands until goal is reached.

[ahundt]

Thanks! that fixed the segfault. Now having following issue Error sending packet to KUKA iiwa: ret = -1, len = 144 [ INFO] [1486491078.220977800]: KukaLBRiiwaROSPlugin in unsupported mode! Looks like I have to publish /interaction_mode? Can't the driver just be set to read without choosing mode?

That's potentially a good reasonable default... Would need to be added to both the java code and the ros code.

Despite the errors being printed here it should still be reading joint data here.

I tried publishing /interaction_mode to MoveArmJointServo. It looks like the driver commands joint positions continuously. Is my interpretation correct?

Yes, also if you send a single set of joint angles it should just stay there.

Why not latch onto last position (or last read position) until new position is commanded?

I believe it does that, what problem are you seeing? It should command the last seen joint position repeatedly if it gets no updates.

Some of our applications require real time responses to changes detected by vision, so simply sending/following a trajectory would mean we couldn't meet those requirements. Stubs are present to handle such a mode but they aren't implemented right now.

One option is an intermediate node that translates that behavior because if you want to approximate a cartesian path in joint space you need to do frequent updates. I believe we have code that does this in https://github.com/jhu-lcsr/costar_stack which also integrates moveit. @cpaxton may know exactly where to look.

You could either use that package set or use it for inspiration.

I'm not sure if I've answered your question yet... :-) let me know and I'll dig a bit more.

[ahundt]

Also was the logger initialization the missing piece of the puzzle?

[ahundt]

I made the following pull request to try and fix the crash you described: https://github.com/ahundt/grl/pull/122

[ahundt]

You may also want to have a look at https://github.com/ahundt/grl/pull/106 which integrates with https://github.com/ros-industrial-consortium/majorana/issues/3, and I believe fixes some of the problems. However, there are merge conflicts and added dependencies I was hoping to make optional that aren't yet resolved.

In particular it seems he made that default case only appear in debug mode: https://github.com/ahundt/grl/pull/106/files#diff-0d01df256a07b23e2a61068455a0d2beR470

            if (debug) {
                ROS_INFO("KukaLBRiiwaROSPlugin in unsupported mode!");
            }

[ahundt]

What we could do to fix this in the ROS driver is at construction time set the mode to default to the StartArm state, which would lead to this code path in KukaLBRiiwaROSPlugin.hpp.

That would stop it printing the errors, and might need to fix up some other lower level steps but I believe it could be made to also allow the current arm position to be read off of the FRI driver.

On the other hand, this problem may strangely be somewhat desirable and the error message just needs to be improved. We could change the error message to indicate that the driver is most likely waiting on an "interaction_mode" message like MoveArmJointServo. In other words give clear instructions for what needs to happen to actually activate the system.

[tdinesh]

logger initialization was the issue. I will test #122 shortly.

I did try StartArm state, but the GRL_Driver complained it is an unsupported Mode.

I like the idea of initializing with StartArm state as default. That way we can atleast read the arm positions.

[tdinesh]

Can the mode_callback be a service instead of a topic callback? Right know there is no way getting an acknowledgement back.

[ahundt]

could add a mode_service perhaps? I'd like to avoid breaking backwards compatibility.

For reference in the timeline https://github.com/ahundt/grl/pull/124 is a new logger crash fix that should be better than the previous one.

[tdinesh]

mode_service sounds good. I will create a pull-request (also keeping the topic callback for backwards compatibility) #124 looks good.

[tdinesh]

That's potentially a good reasonable default... Would need to be added to both the java code and the ros code. Despite the errors being printed here it should still be reading joint data here.

https://github.com/ahundt/grl/blob/master/src/java/grl/src/grl/driver/GRL_Driver.java#L508. Does not handle case grl.flatbuffer.ArmState.StartArm currently.

Also, joint data is not getting published on /joint_states (in MoveArmJointServo too).

[ahundt]

https://github.com/ahundt/grl/pull/125 adds StartArm as an option in the Java code, initializes it in the Java Driver, and just does a hold postion command and prints that the connection is established. Not tested yet but it does compile.

[tdinesh]

This thread is getting long. I'll start new issues as they come along. Closing this one.

I am close to getting the driver working in Java mode with moveit. Will start FRI testing after that.