Closed osrf-migration closed 7 years ago
Original comment by Nate Koenig (Bitbucket: Nathan Koenig).
Issue #21 was marked as a duplicate of this issue.
Original comment by kapoor_amita@yahoo.com (Bitbucket: Amita94).
Attached is the roslaunch log and rqt_graph image when val falls (90% of time)
Script started on Monday 21 November 2016 10:54:55 PM IST ... logging to /home/am/.ros/log/6b65e308-b00f-11e6-af54-685b35a00088/roslaunch-am-Macmini-10908.log Checking log directory for disk usage. This may take awhile. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB. ]2;/opt/ros/indigo/share/srcsim/launch/qual1.launch [1mstarted roslaunch server http://am-Macmini:35969/[0m
PARAMETERS
NODES / IHMCValkyrieROSAPI (ihmc_ros_java_adapter/gradlew) control_py_am_Macmini_10908_6429243349245551040 (robonet_tools/control) controller_manager_am_Macmini_10908_1874398186530051490 (val_deploy/delayed_roslaunch.sh) controller_spawner_am_Macmini_10908_2491801057726095336 (controller_manager/spawner) gazebo (gazebo_ros/gzserver) gazebo_gui (gazebo_ros/gzclient) robot_state_publisher (robot_state_publisher/state_publisher) smtcore_am_Macmini_10908_3371504248313502897 (shared_memory_transport/smtcore) startup_robot (srcsim/init_robot.sh) urdf_spawner_am_Macmini_10908_1798110691714769919 (val_deploy/delayed_roslaunch.sh)
auto-starting new master [1mprocess[master]: started with pid [10920][0m [1mROS_MASTER_URI=http://localhost:11311[0m ]2;/opt/ros/indigo/share/srcsim/launch/qual1.launch http://localhost:11311 [1msetting /run_id to 6b65e308-b00f-11e6-af54-685b35a00088[0m [1mprocess[rosout-1]: started with pid [10933][0m started core service [/rosout] [1mprocess[smtcore_am_Macmini_10908_3371504248313502897-2]: started with pid [10952][0m [1mprocess[control_py_am_Macmini_10908_6429243349245551040-3]: started with pid [10958][0m [1mprocess[controller_manager_am_Macmini_10908_1874398186530051490-4]: started with pid [10959][0m delaying: 4 seconds [1mprocess[gazebo-5]: started with pid [10961][0m [1mprocess[gazebo_gui-6]: started with pid [10965][0m [1mprocess[urdf_spawner_am_Macmini_10908_1798110691714769919-7]: started with pid [10969][0m delaying: 6 seconds [1mprocess[robot_state_publisher-8]: started with pid [10971][0m [1mprocess[IHMCValkyrieROSAPI-9]: started with pid [10972][0m [1mprocess[controller_spawner_am_Macmini_10908_2491801057726095336-10]: started with pid [10976][0m wait 60s for system to load [1mprocess[startup_robot-11]: started with pid [10996][0m Log file /home/am/.log already exists, proceeding... Log file /home/am/.log already exists, proceeding... Gazebo multi-robot simulator, version 7.4.0 Copyright (C) 2012-2016 Open Source Robotics Foundation. Released under the Apache 2 License. http://gazebosim.org
[0m[ INFO] [1479749096.239155720]: Finished loading Gazebo ROS API Plugin.[0m [1;32m[Msg] [0m[1;32mWaiting for master.[0m[1;32m [0m[1;32m[0m[0m[ INFO] [1479749096.240310704]: waitForService: Service [/gazebo/set_physics_properties] has not been advertised, waiting...[0m [1;32m[Msg] [0m[1;32mConnected to gazebo master @ [0m[1;32mhttp://127.0.0.1:11345[0m[1;32m [0m[1;32m[0m[1;32m[Msg] [0m[1;32mPublicized address: [0m[1;32m192.168.1.3[0m[1;32m [0m[INFO] [WallTime: 1479749096.518506] [0.000000] Controller Spawner: Waiting for service controller_manager/load_controller [1m> Loading[22m[1m > buildSrc[22m[1m > settings[22m[11D[0K[0m[1;32m[0m[0m[ INFO] [1479749097.601352082, 0.022000000]: waitForService: Service [/gazebo/set_physics_properties] is now available.[0m [0m[ INFO] [1479749097.677227218, 0.088000000]: Physics dynamic reconfigure ready.[0m [1m > Resolving dependencies ':runtime'[22m[36D[0K[1m > Resolving dependencies 'detachedConfiguration1'[22m[50D[0K[1m > :buildSrc:compileJava[22m[24D[0K[20D[0K:buildSrc:compileJava [1m> Loading > buildSrc[22m[20D[0K[1A[22C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[32C [1m> Loading > buildSrc[22m[1m > :buildSrc:compileGroovy[22m[1m > Resolving dependencies ':compi[22m[33D[0K[26D[0K[20D[0K:buildSrc:compileGroovy [1m> Loading > buildSrc[22m[20D[0K[1A[24C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[34C [1m> Loading > buildSrc[22m[1m > :buildSrc:processResources[22m[29D[0K[20D[0K:buildSrc:processResources [1m> Loading > buildSrc[22m[20D[0K[1A[27C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[37C [1m> Loading > buildSrc[22m[1m > :buildSrc:classes[22m[20D[0K[20D[0K:buildSrc:classes [1m> Loading > buildSrc[22m[20D[0K[1A[18C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[28C [1m> Loading > buildSrc[22m[1m > :buildSrc:jar[22m[16D[0K[20D[0K:buildSrc:jar [1m> Loading > buildSrc[22m[20D[0K[1A[14C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[24C [1m> Loading > buildSrc[22m[1m > :buildSrc:assemble[22m[21D[0K[20D[0K:buildSrc:assemble [1m> Loading > buildSrc[22m[20D[0K[1A[19C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[29C [1m> Loading > buildSrc[22m[1m > :buildSrc:compileTestJava[22m[28D[0K[20D[0K:buildSrc:compileTestJava [1m> Loading > buildSrc[22m[20D[0K[1A[26C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[36C [1m> Loading > buildSrc[22m[1m > :buildSrc:compileTestGroovy[22m[30D[0K[20D[0K:buildSrc:compileTestGroovy [1m> Loading > buildSrc[22m[20D[0K[1A[28C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[38C [1m> Loading > buildSrc[22m[1m > :buildSrc:processTestResources[22m[33D[0K[20D[0K:buildSrc:processTestResources [1m> Loading > buildSrc[22m[20D[0K[1A[31C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[41C [1m> Loading > buildSrc[22m[1m > :buildSrc:testClasses[22m[24D[0K[20D[0K:buildSrc:testClasses [1m> Loading > buildSrc[22m[20D[0K[1A[22C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[32C [1m> Loading > buildSrc[22m[1m > :buildSrc:test[22m[17D[0K[20D[0K:buildSrc:test [1m> Loading > buildSrc[22m[20D[0K[1A[15C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[25C [1m> Loading > buildSrc[22m[1m > :buildSrc:check[22m[18D[0K[20D[0K:buildSrc:check [1m> Loading > buildSrc[22m[20D[0K[1A[16C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[26C [1m> Loading > buildSrc[22m[1m > :buildSrc:build[22m[18D[0K[20D[0K:buildSrc:build [1m> Loading > buildSrc[22m[20D[0K[1A[16C[33mUP-TO-DATE[39m [1m> Loading > buildSrc[22m[20D[0K[1A[26C [1m> Loading > buildSrc[22m[11D[0K[1m > settings[22m[11D[0K[7D[1mConfiguring[22m[1m > 0/1 projects[22m[1m > root project[22m[1m > Resolving dependencies ':classpat[22m[36D[0Kstarting: roslaunch val_deploy val_control_sim.launch controller_manager_looprate:=500 __name:=controller_manager_am_Macmini_10908_1874398186530051490 __log:=/home/am/.ros/log/6b65e308-b00f-11e6-af54-685b35a00088/controller_manager_am_Macmini_10908_1874398186530051490-4.log [1m > Resolving dependencies ':ros'[22m... logging to /home/am/.ros/log/6b65e308-b00f-11e6-af54-685b35a00088/roslaunch-am-Macmini-11267.log Checking log directory for disk usage. This may take awhile. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB. ]2;/opt/nasa/indigo/share/val_deploy/launch/val_control_sim.launch [1mstarted roslaunch server http://am-Macmini:36884/[0m
PARAMETERS
NODES / controller_manager_am_Macmini_11267_3514388487941572722 (val_controller_manager_rtt/controller_exec)
[1mROS_MASTER_URI=http://localhost:11311[0m ]2;/opt/nasa/indigo/share/val_deploy/launch/val_control_sim.launch http://localhost:11311 core service [/rosout] found [1mprocess[controller_manager_am_Macmini_11267_3514388487941572722-1]: started with pid [11288][0m [32D[0K[15D[0K[12D[1m1/1 projects[22m[15D[0K[13D[0K[1m> Building 0%[22m[1m > :parseYaml[22m[13D[0K[13D[0K:parseYaml [1m> Building 0%[22m[13D[0K[1A[11C[33mUP-TO-DATE[39m [1m> Building 0%[22m[13D[0K[1A[21C [1m> Building 0%[22m[2D[1m50%[22m[1m > :runJavaDelegate[22m2016-11-21 22:55:01 localhost ncl_cpp: [NOTICE ] [gov.nasa.HardwareInterface.makeHandle] Robot Hardware has Initialized. Ready to start loading Controllers.
[INFO] [WallTime: 1479749101.652679] [4.037000] Controller Spawner: Waiting for service controller_manager/switch_controller
[INFO] [WallTime: 1479749101.654604] [4.039000] Controller Spawner: Waiting for service controller_manager/unload_controller
[INFO] [WallTime: 1479749101.656461] [4.041000] Loading controller: ihmc_valkyrie_control_java_bridge
[0m[ INFO] [1479749101.704968226, 4.091000000]: Starting JVM with arguments: -Djava.class.path=ValkyrieController.jar -XX:+UseSerialGC -Xmx4g -Xms4g -XX:NewSize=3g -XX:MaxNewSize=3g -XX:CompileThreshold=1000 -verbosegc -Djava.library.path=lib/[0m
Starting Java VM from path /home/am/valkyrie
[33D[0K:runJavaDelegate
[1m> Building 50% > :runJavaDelegate[22m[33D[0KLoading robot model from: 'models/val_description/sdf/valkyrie_sim.sdf'
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[71C
[1m> Building 50% > :runJavaDelegate[22mStarted Java VM: success
starting: roslaunch val_gazebo spawn_urdf_model_from_robot_description.launch modelName:=valkyrie zOffset:=1.25 __name:=urdf_spawner_am_Macmini_10908_1798110691714769919 __log:=/home/am/.ros/log/6b65e308-b00f-11e6-af54-685b35a00088/urdf_spawner_am_Macmini_10908_1798110691714769919-7.log
[31m[ERROR] [1479749102.135462500, 4.519000000]: Nov 21, 2016 10:55:01 PM us.ihmc.valkyrieRosControl.ValkyrieAffinity
PARAMETERS
NODES / urdf_spawner (gazebo_ros/spawn_model)
[1mROS_MASTER_URI=http://localhost:11311[0m
]2;/opt/nasa/indigo/share/val_gazebo/launch/spawn_urdf_model_from_robot_description.launch http://localhost:11311
core service [/rosout] found
[1mprocess[urdf_spawner-1]: started with pid [11429][0m
[33D[0K[INFO] (NetworkParameters.java:38): Looking for network parameters in network parameters file at /opt/ros/indigo/share/ihmc_valkyrie_ros/configurations/IHMCNetworkParametersSim.ini
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[180C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[INFO] (NetworkParameters.java:42): Found Network parameters file at /opt/ros/indigo/share/ihmc_valkyrie_ros/configurations/IHMCNetworkParametersSim.ini
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[152C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[INFO] (NetworkParameters.java:69): Looking for network parameters in environment variables
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[91C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[INFO] (NetworkParameters.java:70): Environment variables will override entries in the network parameters file.
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[111C
[1m> Building 50% > :runJavaDelegate[22mspawn_model script started
[INFO] [WallTime: 1479749103.066184] [0.000000] Loading model xml from ros parameter
[INFO] [WallTime: 1479749103.094208] [5.463000] Waiting for service /gazebo/spawn_urdf_model
[INFO] [WallTime: 1479749103.097863] [5.470000] Calling service /gazebo/spawn_urdf_model
[33D[0KConnecting to controller using TCP on localhost
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[47C
[1m> Building 50% > :runJavaDelegate[22m[1;33mWarning [parser.cc:778][0m XML Element[imuTransform], child of element[sensor] not defined in SDF. Ignoring[imuTransform]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[node], child of element[sensor] not defined in SDF. Ignoring[node]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[api], child of element[sensor] not defined in SDF. Ignoring[api]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[port], child of element[sensor] not defined in SDF. Ignoring[port]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[imuTransform], child of element[sensor] not defined in SDF. Ignoring[imuTransform]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[node], child of element[sensor] not defined in SDF. Ignoring[node]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[api], child of element[sensor] not defined in SDF. Ignoring[api]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[port], child of element[sensor] not defined in SDF. Ignoring[port]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[sensor_number], child of element[sensor] not defined in SDF. Ignoring[sensor_number]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[node], child of element[sensor] not defined in SDF. Ignoring[node]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[api], child of element[sensor] not defined in SDF. Ignoring[api]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[sensor_number], child of element[sensor] not defined in SDF. Ignoring[sensor_number]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[node], child of element[sensor] not defined in SDF. Ignoring[node]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[api], child of element[sensor] not defined in SDF. Ignoring[api]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[imuTransform], child of element[sensor] not defined in SDF. Ignoring[imuTransform]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[node], child of element[sensor] not defined in SDF. Ignoring[node]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[api], child of element[sensor] not defined in SDF. Ignoring[api]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[1;33mWarning [parser.cc:778][0m XML Element[port], child of element[sensor] not defined in SDF. Ignoring[port]. You may have an incorrect SDF file, or an sdformat version that doesn't support this element.
[33D[0K[WARN] YoVariable: desiredMomentumRateLinearX is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[85C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[WARN] YoVariable: desiredMomentumRateLinearY is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[85C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[WARN] YoVariable: desiredMomentumRateLinearZ is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[85C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[WARN] YoVariable: achievedMomentumRateLinearX is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[86C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[WARN] YoVariable: achievedMomentumRateLinearY is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[86C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[WARN] YoVariable: achievedMomentumRateLinearZ is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[86C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[WARN] YoVariable: residualRootJointForceX is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[82C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[WARN] YoVariable: residualRootJointForceY is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[82C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[WARN] YoVariable: residualRootJointForceZ is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[82C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[WARN] YoVariable: residualRootJointTorqueX is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[83C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[WARN] YoVariable: residualRootJointTorqueY is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[83C
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[WARN] YoVariable: residualRootJointTorqueZ is getting created with a null registry
[1m> Building 50% > :runJavaDelegate[22m[33D[0K[1A[83C
[1m> Building 50% > :runJavaDelegate[22m[0m[ INFO] [1479749104.454163884, 5.569000000]: Camera Plugin (robotNamespace = /), Info: Using the 'robotNamespace' param: '/'[0m
[0m[ INFO] [1479749104.454444886, 5.569000000]: Camera Plugin (robotNamespace = /), Info: Using the 'robotNamespace' param: '/'[0m
[0m[ INFO] [1479749104.460090991, 5.569000000]: Camera Plugin (ns = /)
#
#
#
#
pure virtual method called terminate called without an active exception shutting down processing monitor... ... shutting down processing monitor complete [1mdone[0m [31m[controller_spawner_am_Macmini_10908_2491801057726095336-10] escalating to SIGTERM[0m [31m[IHMCValkyrieROSAPI-9] escalating to SIGTERM[0m [31m[gazebo-5] escalating to SIGTERM[0m [31m[controller_spawner_am_Macmini_10908_2491801057726095336-10] escalating to SIGKILL[0m [31m[IHMCValkyrieROSAPI-9] escalating to SIGKILL[0m [rosout-1] killing on exit [master] killing on exit [31mShutdown errors:
Script done on Monday 21 November 2016 10:58:04 PM IST
Original comment by Jordan Lack (Bitbucket: jordanlack).
@Amita94 it looks like this is a separate issue than what this ticket was initially opened up to address. It's a bit difficult to parse through that output, but it sounds like you're having a problem with Valkyrie actually falling down in Gazebo, where this issue was created due to a crashing issue.
I am currently working on a release that includes a number of features including a fix for the crashing issue reported by a few users here(see top of this thread).
Original comment by Franky Lau (Bitbucket: frankylau).
hi @JordanLack, sorry that I am not going to hurry you up, but just want to make it clear.
is the new release will address the 2 physical cores problem as @dljsjr described?
thanks.
Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).
Ok, everyone, IHMC has released a new version of their controller (version 0.8.1), and I've uploaded a new valkyrie_controller.tar.gz (step 10 in installation instructions), so please delete the ~/valkyrie
folder and download the new one and extract it in its place. This should fix the issues with dual-core CPU's.
Can some folks test this out to see if it helps?
Original comment by Nuttaworn Sujumnong (Bitbucket: nsujumnong).
Excellent. The controller works on my computer now. Also, I don't get hyperthreading error message any longer, which is kinda cool. Thank you!
Original comment by dan (Bitbucket: dan77062).
It no longer has the IndexOutOfBoundsException, so the patch has solved the 2 core problem-- thanks! Unfortunately, the robot still collapses. Since I think that is due to other reasons, I put a comment along with output into issue #9 I think this issue (2 cores) is resolved.
Original comment by Adam Allevato (Bitbucket: Kukanani).
The new controller definitely seems to have improved software stability. Good work.
Original comment by dan (Bitbucket: dan77062).
The two core problem is solved-- thanks!
Original report (archived issue) by dan (Bitbucket: dan77062).
The original report had attachments: master.log, roslaunch-Mingo-Mtn-Robotics-24044.log, control_py_Mingo_Mtn_Robotics_24044_828331878780894790-3-stdout.log, roslaunch-Mingo-Mtn-Robotics-24096.log, rosout.log, rosout-1-stdout.log
The controller manager crashes as it tries to initialize the hardware interface. This continues after updating with the fix for the $HOME/valkyrie directory missing issue (issue #4). See the log below.
I am wondering if it is a race condition. It will load when gazebo_gui seg faults (that happens about 20% of the time). Log of that follows the controller manager crash log.
Here is the log of the controller manager crashing:
Here the controller manager loads after gazebo_gui seg faults: