search

microsoft / AirSim

Open source simulator for autonomous vehicles built on Unreal Engine / Unity, from Microsoft AI & Research
https://microsoft.github.io/AirSim/
Other
16.45k stars 4.58k forks source link

MavlinkTest Proxy between PX4 and Airsim not working #3376

Closed maxystory closed 2 years ago

maxystory commented 3 years ago

Question

What's your question?

I'm trying to connect a HIL Pixhawk4 to Airsim Unreal Engine through MavlinkTest UDP. The Pixhawk4 is version 1.11.3, and Airsim is running on VS19 and a windows 10 laptop. The reason being is I would like to add sensor error through MavlinkTest.

Include context on what you are trying to achieve

  1. First I open the default blocks unreal environment.
  2. Then I open developer command prompt VS19, and ran mavlink, trying to connect serial (Pixhawk4) to proxy. 
    E:\AirSim-master\MavLinkCom\MavLinkTest\build\x64\Debug>mavlinktest -serial -proxy:127.0.0.1:14560
Auto Selecting COM port: MindPX (COM1)
Connecting to serial port COM1, baudrate=115200
Connecting to UDP Proxy address 127.0.0.1:14560
Downloading drone parameters so we know how to control it properly...
PX4_CUSTOM_MAIN_MODE_MANUAL
MavLinkConnectionImpl: Error handling message 4 on connection 'drone', details: MavLinkConnectionImpl: Error sending message on connection 'proxy', details: UdpClientPort socket send failed with error: 10038
STATUS: sev=2, 'Connection to ground station lost'
STATUS: sev=2, 'system power unavailable'
STATUS: sev=6, 'Data link regained'
Ready...
### command 176 result: MAV_RESULT_ACCEPTEDSystem status:
onboard sensors present:
MAV_SYS_STATUS_SENSOR_DIFFERENTIAL_PRESSURE - 0x10 differential pressure
MAV_SYS_STATUS_SENSOR_GPS - 0x20 GPS
MAV_SYS_STATUS_SENSOR_VISION_POSITION - 0x80 computer vision position
onboard sensors enabled:
CPU load 35
battery voltage 0 millivolts
battery current 25966 milliamps
drop_rate_comm = 100
errors_comm = 0
battery_remaining = 0
landed_state=MAV_LANDED_STATE_ON_GROUND
vtol_state=0
Connected:
Version=3
Type=MAV_TYPE_QUADROTOR-  Quadrotor
Autopilot=MAV_AUTOPILOT_PX4, PX4 Autopilot - http://pixhawk.ethz.ch/px4/
State=MAV_STATE_STANDBY - System is grounded and on standby. It can be launched any time.
Base mode:
MAV_MODE_FLAG_STABILIZE_ENABLED - system stabilizes electronically its attitude (and optionally position). It needs however further control inputs to move around.
MAV_MODE_FLAG_HIL_ENABLED - hardware in the loop simulation. All motors / actuators are blocked, but internal software is full operational.
MAV_MODE_FLAG_MANUAL_INPUT_ENABLED - remote control input is enabled.
PX4_CUSTOM_MAIN_MODE_MANUAL
VEHICLE SYSTEM ID: 1
VEHICLE COMPONENT ID: 1
  3. It did not work with error sending message to udp server, so then I tried to use -server instead of -proxy, which worked. 
    E:\AirSim-master\MavLinkCom\MavLinkTest\build\x64\Debug>mavlinktest -serial -server:127.0.0.1:14560
Auto Selecting COM port: MindPX (COM1)
Connecting to serial port COM1, baudrate=115200
Connecting to UDP Server address 127.0.0.1:14560
Downloading drone parameters so we know how to control it properly...
PX4_CUSTOM_MAIN_MODE_MANUAL
Ready...
### command 176 result: MAV_RESULT_ACCEPTEDSystem status:
onboard sensors present:
MAV_SYS_STATUS_SENSOR_DIFFERENTIAL_PRESSURE - 0x10 differential pressure
MAV_SYS_STATUS_SENSOR_GPS - 0x20 GPS
MAV_SYS_STATUS_SENSOR_VISION_POSITION - 0x80 computer vision position
onboard sensors enabled:
CPU load 180
battery voltage 0 millivolts
battery current 1023 milliamps
drop_rate_comm = 267
errors_comm = 1023
battery_remaining = 0
landed_state=MAV_LANDED_STATE_ON_GROUND
vtol_state=0
Connected:
Version=3
Type=MAV_TYPE_QUADROTOR-  Quadrotor
Autopilot=MAV_AUTOPILOT_PX4, PX4 Autopilot - http://pixhawk.ethz.ch/px4/
State=MAV_STATE_STANDBY - System is grounded and on standby. It can be launched any time.
Base mode:
MAV_MODE_FLAG_STABILIZE_ENABLED - system stabilizes electronically its attitude (and optionally position). It needs however further control inputs to move around.
MAV_MODE_FLAG_HIL_ENABLED - hardware in the loop simulation. All motors / actuators are blocked, but internal software is full operational.
MAV_MODE_FLAG_MANUAL_INPUT_ENABLED - remote control input is enabled.
PX4_CUSTOM_MAIN_MODE_MANUAL
VEHICLE SYSTEM ID: 1
VEHICLE COMPONENT ID: 1
  4. Then i click play on unreal engine to try to connect airsim with the mavlinktest. However, there is an error relating to the mavlink message length. The message length 37 doesn't match message length 36.

image

On the developer VS19 cmd, it looks like this:

E:\AirSim-master\MavLinkCom\MavLinkTest\build\x64\Debug>mavlinktest -serial -server:127.0.0.1:14560
Auto Selecting COM port: MindPX (COM1)
Connecting to serial port COM1, baudrate=115200
Connecting to UDP Server address 127.0.0.1:14560
Downloading drone parameters so we know how to control it properly...
    PX4_CUSTOM_MAIN_MODE_MANUAL
Ready...
### command 176 result: MAV_RESULT_ACCEPTEDSystem status:
    onboard sensors present:
    MAV_SYS_STATUS_SENSOR_DIFFERENTIAL_PRESSURE - 0x10 differential pressure
    MAV_SYS_STATUS_SENSOR_GPS - 0x20 GPS
    MAV_SYS_STATUS_SENSOR_VISION_POSITION - 0x80 computer vision position
    onboard sensors enabled:
    CPU load 180
    battery voltage 0 millivolts
    battery current 1023 milliamps
    drop_rate_comm = 267
    errors_comm = 1023
    battery_remaining = 0
    landed_state=MAV_LANDED_STATE_ON_GROUND
    vtol_state=0
Connected:
    Version=3
    Type=MAV_TYPE_QUADROTOR-  Quadrotor
    Autopilot=MAV_AUTOPILOT_PX4, PX4 Autopilot - http://pixhawk.ethz.ch/px4/
    State=MAV_STATE_STANDBY - System is grounded and on standby. It can be launched any time.
    Base mode:
    MAV_MODE_FLAG_STABILIZE_ENABLED - system stabilizes electronically its attitude (and optionally position). It needs however further control inputs to move around.
    MAV_MODE_FLAG_HIL_ENABLED - hardware in the loop simulation. All motors / actuators are blocked, but internal software is full operational.
    MAV_MODE_FLAG_MANUAL_INPUT_ENABLED - remote control input is enabled.
    PX4_CUSTOM_MAIN_MODE_MANUAL
    VEHICLE SYSTEM ID: 1
    VEHICLE COMPONENT ID: 1
STATUS: sev=2, 'Connection to ground station lost'
STATUS: sev=2, 'system power unavailable'
STATUS: sev=6, 'Data link regained'
STATUS: sev=0, 'BARO #0 failed:  TIMEOUT!'
MavLinkConnectionImpl: Error handling message 111 on connection 'drone', details: MavLinkConnectionImpl: Error sending message on connection 'server', details: UdpClientPort socket send failed with error: 10038
  1. Then, I tried to just connect the MavlinkTest to the airsim unreal without the pixhawk4, which works. Which means both pixhawk4 and the airsim unreal can connect to MavlinkTest. However, their message length are different from each other. How can this issue be fixed? 
    E:\AirSim-master\MavLinkCom\MavLinkTest\build\x64\Debug>mavlinktest -server:127.0.0.1:14560
Connecting to UDP Server address 127.0.0.1:14560
isLocalControlSupported failed: Two second timeout waiting for response to mavlink command MAV_CMD_REQUEST_AUTOPILOT_CAPABILITIES
Downloading drone parameters so we know how to control it properly...
Ready...
System status:
onboard sensors present:
onboard sensors enabled:
MAV_SYS_STATUS_SENSOR_3D_ACCEL - 0x02 3D accelerometer
MAV_SYS_STATUS_SENSOR_3D_MAG - 0x04 3D magnetometer
CPU load 0
battery voltage 0 millivolts
battery current 0 milliamps
drop_rate_comm = 0
errors_comm = 0
battery_remaining = -1
Connected:
Version=3
Type=MAV_TYPE_GCS-  Operator control unit / ground control station
Autopilot=MAV_AUTOPILOT_GENERIC, Generic autopilot, full support for everything
State=MAV_STATE_UNINIT - Uninitialized system, state is unknown.
Base mode:
Custom mode=0    VEHICLE SYSTEM ID: 1
VEHICLE COMPONENT ID: 1

Many thanks for your suggestions and help!

Context details

The AirSim version is the latest 1.4.0 windows version. The unreal version is 4.23.1. The settings.json are 

{
    "SettingsVersion": 1.2,
    "SimMode": "Multirotor",
    "Vehicles": {
        "PX4": {
            "LocalHostIp": "127.0.0.1",
            "UdpIp": "127.0.0.1",
            "UdpPort": 14560,
            "ControlPort": 14560,
            "UseSerial": false,
            "UsingTcp": false,
            "VehicleType": "PX4Multirotor",
            "VehicleCompID": 1,
            "VehicleSysID": 1
        }
    }
}

Include details of what you already did to find answers

jonyMarino commented 3 years ago

Hi @maxystory, and welcome! Thank you for your report. I think you are using unreal engine 4.25, not 4.23, which is not compatible with the latest version of AirSim. We will take a look at this.

maxystory commented 3 years ago

Hi @jonyMarino, thanks for the reply! After days of trial and error, I found out that the MavlinkTest worked for me on airsim v1.2.0, VS17, running unreal engine 4.18. Like you suggested, I have also tried unreal engine 4.23 and 4.25, and it both didn't work. Perhaps I am doing something incorrect but I hope you guys can look into it for future development.

I think the reason behind is, the udp port setting (in settings.json) is not working properly on unreal engine 4.23. When I opened a UDP server in MavlinkTest (127.0.0.1:14560), I don't see signs of the udp port working. Maybe it self connects? Also, no matter what udp port I enter in the settings.json, it seems to claim it successfully connects to the udp socket. image

I hope this helps you on fixing the issue. Thank you for all airsim's team great work!

p00uya commented 3 years ago

Hi there I can connect raspberry pi to pixhawk with mavproxy according to this link instruction , now is it possible to connect RPI to airsim? I want to use AIRSIM as GCS and read sensor on airsim in HITL form, is it possible??? i check this link and run the code and saw not working

jonyMarino commented 2 years ago

@maxystory Mavlink-router can be used instead of MavlinkTest. @pp00uya it should be possible. Please share the details in a new issue. Closed due to inactivity. Please, feel free to ask for it to be reopened if you wish to continue posting.

AshishSingh-Glitch90 commented 1 year ago

Hi, I am trying to implement flight simulation using Airsim hardware in the loop(HITL), px4 version 1.9.2 on pixhawk 2.4.8 by programming pixhawk using Qgroundcontrol which I was able to work. Now I want to implement the same with the Xbox Controller for which I followed:

https://microsoft.github.io/AirSim/xbox_controller/ and https://docs.px4.io/main/en/simulation/hitl.html

using this "MavLinkTest.exe -serial:*,115200 -proxy:127.0.0.1:14550 -server:127.0.0.1:14570" command I am able to build a connect between the QGroundControl and Proxy but Airsim shows a message to restart the HITL node as shown below:

Screenshot_QgroundControl_UDPconnection

image

My Airsim Json settings are:

{ "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md", "SettingsVersion": 1.2, "SimMode": "Multirotor", "ViewMode": "", "ClockType": "SteppableClock", "Vehicles": { "PX4": { "VehicleType": "PX4Multirotor", "SitlIp": "", "SitlPort": 14560, "UdpIp": "127.0.0.1", "UdpPort": 14570, "UseSerial": false, "LockStep": true, "Parameters": { "NAV_RCL_ACT": 0, "NAV_DLL_ACT": 0, "COM_OBL_ACT": 1, "LPE_LAT": 47.641468, "LPE_LON": -122.140165 } } } }

Do you have any suggestion how I can fix this.

Thanks