search

uzh-rpg / agile_flight

Developing and Comparing Vision-based Algorithms for Vision-based Agile Flight
MIT License
152 stars 55 forks source link

visionenv_sim.launch prints out yellow and red messages #15

Open minghanwei opened 2 years ago

minghanwei commented 2 years ago

Thanks for the great work. When I follow the steps in the instructions: roslaunch envsim visionenv_sim.launch render:=true finally shows 'UnityBridge] Flightmare Unity is connected.' but prints out some warning (in yellow) and red messages. I just would like to check whether it is normal. It says:

** message in yellow **** [PilotParams] Did not create inner controller ''! [Pilot] Did not create bridge 'ROS'. Using debug bridge, register externally!

*message in red *** [dodgeros_gui-4] process has died [pid 47655, exit code -11, cmd /opt/ros/noetic/lib/rqt_gui/rqt_gui -s dodgeros_gui.basic_flight.BasicFlight --args --quad_name kingfisher __name:=dodgeros_gui __log:=/home/minghan/.ros/log/c1d886b4-ab18-11ec-b3ca-07c585af6c36/dodgeros_gui-4.log]. log file: /home/minghan/.ros/log/c1d886b4-ab18-11ec-b3ca-07c585af6c36/dodgeros_gui-4*.log

Could I ask whether these are normal or there is a setting-up issue? Thank you very much.

minghanwei commented 2 years ago

By the way, when I followed all the steps, the uav did not move. I guess there is something wrong in my setup.

yun-long commented 2 years ago

Hi,

there is something wrong with the setup. yellow messages sometimes are fine. but, the red message means the ROS node is failed.

Sheng-Cheng commented 2 years ago

I also get this error. I think the gui is dead and then you can't command the vehicle.

[dodgeros_gui-4] process has died [pid 6730, exit code 1, cmd /opt/ros/noetic/lib/rqt_gui/rqt_gui -s dodgeros_gui.basic_flight.BasicFlight --args --quad_name kingfisher __name:=dodgeros_gui __log:=/home/sheng/.ros/log/0ee9fdd4-abeb-11ec-b44e-712162f9a126/dodgeros_gui-4.log]. log file: /home/sheng/.ros/log/0ee9fdd4-abeb-11ec-b44e-712162f9a126/dodgeros_gui-4*.log

lbfd commented 2 years ago

Can you please post the full terminal output of the error? Before the GUI process dies?

minghanwei commented 2 years ago

I also get this error. I think the gui is dead and then you can't command the vehicle.

[dodgeros_gui-4] process has died [pid 6730, exit code 1, cmd /opt/ros/noetic/lib/rqt_gui/rqt_gui -s dodgeros_gui.basic_flight.BasicFlight --args --quad_name kingfisher __name:=dodgeros_gui __log:=/home/sheng/.ros/log/0ee9fdd4-abeb-11ec-b44e-712162f9a126/dodgeros_gui-4.log]. log file: /home/sheng/.ros/log/0ee9fdd4-abeb-11ec-b44e-712162f9a126/dodgeros_gui-4*.log

minghanwei commented 2 years ago

Here is my terminal output with errors:

auto-starting new master process[master]: started with pid [55150] ROS_MASTER_URI=http://localhost:11311

setting /run_id to 9de44bb0-ac41-11ec-a0f4-833bfc0b1034 process[rosout-1]: started with pid [55160] started core service [/rosout] process[kingfisher/dodgeros_pilot-2]: started with pid [55167] process[kingfisher/viz_face-3]: started with pid [55168] process[dodgeros_gui-4]: started with pid [55169] process[flight_render-5]: started with pid [55170] [ WARN] [1648215836.833028841]: Pilot Config: simple_sim_pilot.yaml [ WARN] [1648215836.833497074]: Agi Param Directory: /home/minghan/icra22_competition_ws/src/agile_flight/dodgedrone_simulation/dodgelib/params [ WARN] [1648215836.833518243]: ROS Param Directory: /home/minghan/icra22_competition_ws/src/agile_flight/envsim/parameters [ INFO] [1648215836.833539656]: Loading Pilot Params from simple_sim_pilot.yaml in /home/minghan/icra22_competition_ws/src/agile_flight/envsim/parameters Loading Pilot parameters from "/home/minghan/icra22_competition_ws/src/agile_flight/envsim/parameters/simple_sim_pilot.yaml" [PilotParams] Did not create inner controller ''! [Pilot] Did not create bridge 'ROS'. Using debug bridge, register externally! [ INFO] [1648215836.835993646]: Loaded pipeline: Estimator: Type: Feedthrough File: "/home/minghan/icra22_competition_ws/src/agile_flight/dodgedrone_simulation/dodgelib/params/feedthrough.yaml" Sampler: Type: Time File: "" Outer Controller: Type: GEO File: "/home/minghan/icra22_competition_ws/src/agile_flight/dodgedrone_simulation/dodgelib/params/geo.yaml" Inner Controller: Type: File: "" Bridge: Type: ROS File: ""

[Pilot] Register external bridge: [ROS Bridge]
which was not active and used. [kingfisher/dodgeros_pilot-2] process has died [pid 55167, exit code -11, cmd /home/minghan/icra22_competition_ws/devel/lib/envsim/visionsim_node name:=dodgeros_pilot log:=/home/minghan/.ros/log/9de44bb0-ac41-11ec-a0f4-833bfc0b1034/kingfisher-dodgeros_pilot-2.log]. log file: /home/minghan/.ros/log/9de44bb0-ac41-11ec-a0f4-833bfc0b1034/kingfisher-dodgeros_pilot-2*.log PluginManager._discover() force discovery of plugins RospkgPluginProvider._find_plugins() crawling for plugins of type 'qt_gui' [dodgeros_gui-4] process has died [pid 55169, exit code -11, cmd /opt/ros/noetic/lib/rqt_gui/rqt_gui -s dodgeros_gui.basic_flight.BasicFlight --args --quad_name kingfisher name:=dodgeros_gui log:=/home/minghan/.ros/log/9de44bb0-ac41-11ec-a0f4-833bfc0b1034/dodgeros_gui-4.log]. log file: /home/minghan/.ros/log/9de44bb0-ac41-11ec-a0f4-833bfc0b1034/dodgeros_gui-4*.log

lbfd commented 2 years ago

The log so far looks ok and we can't reproduce the problem. Could you please list your system details:

minghanwei commented 2 years ago

I have: Ubuntu 20.04.4 LTS, kernal 5.13.0-37-generic processor: Intel® Core™ i7-10700F CPU NVIDIA Corporation TU116 [GeForce GTX 1660 Ti] gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0 python 3.8.10 'pkg-config --modversion eigen3' prints out 3.3.7

Thanks for the help. I am also trying to figure out the issue.

HarukiKozukapenguin commented 2 years ago

@lbfd @yun-long I might have the same problem, and I will show you the error. I typed following commands _roslaunch envsim visionenvsim.launch render:=True (I checked I can launch the following rviz) Screenshot from 2022-04-01 21-16-34 output code is as follows

... logging to /home/myname/.ros/log/86270e1c-b1b5-11ec-9f77-f12aa127d5cc/roslaunch-myname-X299-AORUS-MASTER-5017.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://myname-X299-AORUS-MASTER:40907/

SUMMARY
========

PARAMETERS
 * /kingfisher/dodgeros_pilot/agi_param_dir: /home/myname/icra22...
 * /kingfisher/dodgeros_pilot/camera_config: /home/myname/icra22...
 * /kingfisher/dodgeros_pilot/low_level_controller: Simple
 * /kingfisher/dodgeros_pilot/pilot_config: simple_sim_pilot....
 * /kingfisher/dodgeros_pilot/real_time_factor: 1.0
 * /kingfisher/dodgeros_pilot/render: True
 * /kingfisher/dodgeros_pilot/ros_param_dir: /home/myname/icra22...
 * /kingfisher/dodgeros_pilot/use_bem_propeller_model: False
 * /rosdistro: noetic
 * /rosversion: 1.15.14
 * /use_sim_time: True

NODES
  /
    dodgeros_gui (rqt_gui/rqt_gui)
    flight_render (flightrender/RPG_Flightmare.x86_64)
  /kingfisher/
    dodgeros_pilot (envsim/visionsim_node)
    viz_face (rviz/rviz)

auto-starting new master
process[master]: started with pid [5027]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 86270e1c-b1b5-11ec-9f77-f12aa127d5cc
process[rosout-1]: started with pid [5037]
started core service [/rosout]
process[kingfisher/dodgeros_pilot-2]: started with pid [5040]
process[kingfisher/viz_face-3]: started with pid [5045]
process[dodgeros_gui-4]: started with pid [5046]
process[flight_render-5]: started with pid [5047]
[ WARN] [1648815375.004059488]: Pilot Config:        simple_sim_pilot.yaml
[ WARN] [1648815375.005562413]: Agi Param Directory: /home/myname/icra22_competition_ws/src/agile_flight/dodgedrone_simulation/dodgelib/params
[ WARN] [1648815375.005632329]: ROS Param Directory: /home/myname/icra22_competition_ws/src/agile_flight/envsim/parameters
[ INFO] [1648815375.005667021]: Loading Pilot Params from simple_sim_pilot.yaml in /home/myname/icra22_competition_ws/src/agile_flight/envsim/parameters
Loading Pilot parameters from "/home/myname/icra22_competition_ws/src/agile_flight/envsim/parameters/simple_sim_pilot.yaml"
[PilotParams]  Did not create inner controller ''!
[Pilot]        Did not create bridge 'ROS'.
Using debug bridge, register externally!
[ INFO] [1648815375.014592878]: Loaded pipeline:
Estimator:
Type: Feedthrough
File: "/home/myname/icra22_competition_ws/src/agile_flight/dodgedrone_simulation/dodgelib/params/feedthrough.yaml"
Sampler:
Type: Time
File: ""
Outer Controller:
Type: GEO
File: "/home/myname/icra22_competition_ws/src/agile_flight/dodgedrone_simulation/dodgelib/params/geo.yaml"
Inner Controller:
Type: 
File: ""
Bridge:
Type: ROS
File: ""

[Pilot]        Register external bridge: [ROS Bridge]   
 which was not active and used.
[VisionEnv]    Camera has been added. Skipping the camera configuration.
[UnityBridge]  Initializing ZMQ connection!
[UnityBridge]  Initializing ZMQ connections done!
[VisionEnv]    Flightmare Bridge created.
[UnityBridge]  Trying to Connect Unity.
[.....................................................................................[UnityBridge]  Flightmare Unity is connected.

Then, we type this command. _python3 evaluationnode.py _python runcompetition.py then, we checked _publishcommand function is running by printing command in this function. Then, we run the following command. _rostopic pub /kingfisher/start_navigation stdmsgs/Empty "{}" -1 then we simulator stopped with following commands in the terminal of _roslaunch envsim visionenvsim.launch render:=True command.

[kingfisher/dodgeros_pilot-2] process has died [pid 6186, exit code -11, cmd /home/myname/icra22_competition_ws/devel/lib/envsim/visionsim_node __name:=dodgeros_pilot __log:=/home/myname/.ros/log/fe66330c-b1b6-11ec-9f77-f12aa127d5cc/kingfisher-dodgeros_pilot-2.log].
log file: /home/myname/.ros/log/fe66330c-b1b6-11ec-9f77-f12aa127d5cc/kingfisher-dodgeros_pilot-2*.log

I think this issue happens when we published the following topic. _"/" + quad_name + "/dodgeros_pilot/velocitycommand" (run_competition.py l46) I checked the this topic when I run the last command by following command. _rostopic echo /kingfisher/dodgeros_pilot/velocitycommand Out put is as follows(published twice, and there are no commands after that):

WARNING: no messages received and simulated time is active.
Is /clock being published?
header: 
  seq: 1
  stamp: 
    secs: 1648817103
    nsecs: 714545011
  frame_id: ''
twist: 
  linear: 
    x: 0.0
    y: 0.0
    z: 0.0
  angular: 
    x: 0.0
    y: 0.0
    z: 0.0
---
header: 
  seq: 2
  stamp: 
    secs: 1648817103
    nsecs: 714545011
  frame_id: ''
twist: 
  linear: 
    x: 0.0
    y: 0.0
    z: 0.0
  angular: 
    x: 0.0
    y: 0.0
    z: 0.0
---

So, I think rviz is stopped when rivz receive this topic(/kingfisher/dodgeros_pilot/velocity_command), but I don't know why.

Where rviz receives these commands?

I checked rostopic info /kingfisher/dodgeros_pilot/velocity_command

Type: geometry_msgs/TwistStamped

Publishers: 
 * /agile_pilot_node (http://haru-X299-AORUS-MASTER:35969/)

Subscribers: 
 * /kingfisher/dodgeros_pilot (http://haru-X299-AORUS-MASTER:43761/)
 * /rostopic_7736_1648817110850 (http://haru-X299-AORUS-MASTER:44181/)

Can there subscribers can receive this topic?

Then we checked the following command. rosnode info /kingfisher/dodgeros_pilot Then, we can find the following topic. and this node do not know the type of velocity command. * /kingfisher/dodgeros_pilot/velocity_command [unknown type] This might cause problem?

I run this command in the following bashrc environment.

source /opt/ros/noetic/setup.bash
export ROS_VERSION=noetic
export CATKIN_WS=./icra22_competition_ws
source ~/icra22_competition_ws/devel/setup.bash
export FLIGHTMARE_PATH=~/icra22_competition_ws/src/agile_flight/flightmare

The last command is come from this issue. https://github.com/uzh-rpg/agile_flight/issues/9

I have: Ubuntu 20.04.4 LTS, gcc,g++ 9.4.0 python 3.8.10 'pkg-config --modversion eigen3' prints out 3.3.7

P.S. Even when we change the command for controller as follows, the error do not change. _python run_competition.py --visionbased

k-okada commented 2 years ago

@HarukiKozukapenguin It is working on my environment. First please try with only ROS, without Anaconda, so please remove anaconda settings from ~/.bashrc Then, follow the 1st step of instructions (https://github.com/uzh-rpg/agile_flight/blob/main/README.md)

roslaunch envsim visionenv_sim.launch render:=True
# Using the GUI, press Arm & Start to take off.

and send velocity

rostopic pub -r 10 /kingfisher/dodgeros_pilot/velocity_command geometry_msgs/TwistStamped "header:
  seq: 0
  stamp:
    secs: 0
    nsecs: 0
  frame_id: ''
twist:
  linear:
    x: 0.0
    y: 0.0
    z: 1.0
  angular:
    x: 0.0
    y: 0.0
    z: 0.0"

You can see the robot is going up

Then you start different terminal, source cra22_competition_ws/deve/setup.bash , conda activate agileflight0, then start the python code.

Cc: @Takuzumi240, @tongtybj

tongtybj commented 2 years ago

@k-okada @HarukiKozukapenguin

I aslo succeeded to launch visionenv_sim.launch in Docker env like this: Screenshot from 2022-04-06 03-06-39

I succeeded to use following topics to control the drone:

HarukiKozukapenguin commented 2 years ago

@k-okada Thank you for helping me. I launched by following instruction.

roslaunch envsim visionenv_sim.launch render:=True The simulator launched as expected. But, when we send velocity topic by following command, the process of launch file is dead with the following command.

rostopic pub -r 10 /kingfisher/dodgeros_pilot/velocity_command geometry_msgs/TwistStamped "header:
  seq: 0
  stamp:
    secs: 0
    nsecs: 0
  frame_id: ''
twist:
  linear:
    x: 0.0
    y: 0.0
    z: 1.0
  angular:
    x: 0.0
    y: 0.0
    z: 0.0"
[kingfisher/dodgeros_pilot-2] process has died [pid 3082, exit code -11, cmd /home/myplace/icra22_competition_ws/devel/lib/envsim/visionsim_node __name:=dodgeros_pilot __log:=/home/myplace/.ros/log/61ddaeb2-b57f-11ec-a1f2-dbf059b0747b/kingfisher-dodgeros_pilot-2.log].
log file: /home/myplace/.ros/log/61ddaeb2-b57f-11ec-a1f2-dbf059b0747b/kingfisher-dodgeros_pilot-2*.log

So, I will reinstall this environment.

P.S. My ~/.bashrc environment is as follows.

source /opt/ros/noetic/setup.bash
export ROS_VERSION=noetic
export CATKIN_WS=./icra22_competition_ws
source ~/icra22_competition_ws/devel/setup.bash
export FLIGHTMARE_PATH=~/icra22_competition_ws/src/agile_flight/flightmare

Is it OK to write these commands to ~/.bashrc?

P.S. I reinstall this environment, but that does not changed.

HarukiKozukapenguin commented 2 years ago

When we push "Arm bridge" , then this bug is cured, but I don't know why.

emanuelazfernandes commented 2 years ago

I don't know if you fixed it, but the first error I was getting with: [dodgeros_gui-4] process has died [pid 47655, exit code -11, cmd /opt/ros/noetic/lib/rqt_gui/rqt_gui -s was solved by conda deactivate. The second one, I suggest removing the export ROS_VERSION variable from .bashrc, as it conflicts with the one already present in /opt/ros/noetic/setup.bash Finally, I suggest also putting the full path in the FLIGHTMARE_PATH variable, e.g /home/yourusername, so the last 3 lines of my ~/.bashrc look like this:

source /opt/ros/noetic/setup.bash source /home/myusername/catkin_workspaces/icra22_competition_ws/devel/setup.bash export FLIGHTMARE_PATH=/home/myusername/catkin_workspaces/icra22_competition_ws/src/agile_flight/flightmare

HarukiKozukapenguin commented 2 years ago

I have already fixed it, and I now run in python