search

unl-nimbus-lab / pymavswarm

Python library used to safely control drone swarms and drone fleets with MAVLink
https://pymavswarm.readthedocs.io
GNU General Public License v3.0
18 stars 5 forks source link

Integrate Software-In-The-Loop Configuration #63

Open evan-palmer opened 2 years ago

evan-palmer commented 2 years ago

Is your feature request related to a problem? Please describe. Users are currently required to have a full hardware setup to be able to test their algorithms and implementation. This can make it difficult to evaluate their software when the environment is not suitable for testing (e.g., winter) or when Users don't have access to hardware platforms for testing.

Describe the solution you'd like Implement support for SITL

evan-palmer commented 2 years ago

It could be useful to integrate configurations for SITL initialization similar to the environment configuration used in OpenAI's safety-gym here

evan-palmer commented 2 years ago

Here is relevant documentation that discusses using other ground control stations with SITL.

DanielAmigo commented 1 year ago

Hello,

first of all, your project is very interesting @evan-palmer, congratulations.

I would like to contribute on its integration with SITL, but I have some doubts about how to approach it.

I don't have much experience on real swarms, and I guess that's what I don't understand. How does it actually communicate with the real drones, PX4s and companion computers? What is physically connected? and to where? I can see two possibilities, tell me if I'm right.

To give you some insight on what I know about swarms, I'll explain the simulation project we've developed last year and why I think your project fits in with it. We developed a decentralised swarm simulation in AirSim using PX4s in SITL, each drone using MavSDK-Python to access the PX4 on its own thread as a companion computer. It worked, but the problem is that the communication we implemented is unrealistic for real drones. We wrote the communication messages in files, taking advantage of the fact that they are all running on the same computer. With this approach we managed to make the following features to work: communication between drones and to detect each other and create a swarm, to design a formation, to move in formation in a synchronized manner, to relocate when a drone ceased to be part of the formation, and to avoid obstacles collaboratively (if a drone detected it and another one did not follow behind it). Looking at your library I think it does make sense to incorporate it in our project, as it is a more realistic and formal communication approach, using Mavlink messages.

Do you have any initial idea of what could be used to make the rest of the drones receive messages in simulation? Maybe with MaxProxy or some app like that? We have also the ability to perform HITL, in case it is useful to migrate from real to simulation.

evan-palmer commented 1 year ago

Thank you for your kind words and interest in contributing @DanielAmigo!

I don't have much experience on real swarms, and I guess that's what I don't understand. How does it actually communicate with the real drones, PX4s and companion computers? What is physically connected? and to where? I can see two possibilities, tell me if I'm right.

  • A typical telemetry antenna to a laptop, which communicates with the N antennas of each PX4 on their respective drones, thus having a centralized swarm. Or, does it need one antenna per PX4?
  • The second one, is the Python process to be executed by each drone's companion computer, connecting it to its PX4, thus having an uncentralized swarm? In that case, how does each PX4 communicate to the rest of the swarm?

You're on the right track with both of these, and the answer is that it depends on how users choose to configure their network. I know that's a pretty unhelpful answer, so I'll talk a bit about how pymavswarm has been designed, how the network was configured for successful hardware tests, and what should work but is untested.

pymavswarm expects to have a network connection to all agents that it should be interacting with, and doesn't necessarily care whether its running on the agents themselves for decentralized swarms or on a GCS for centralized swarms. pymavswarm discovers agents as agents send HEARTBEAT messages on the network. As agents send other state messages, the MavSwarm instance will update the stored state of the agents. Given the network connection to the agents, the MavSwarm instance can also send messages to agents. It is worth noting, however, that only a single MavSwarm instance can be connected to a serial connection, so you would likely anticipate that only one MavSwarm process is running per vehicle (unless there are multiple radios onboard that different instances connect to).

So what this could look like in a centralized swarm configuration is a GCS with a radio or router capable of connecting to all agents in the network and a single Python process. In a decentralized swarm, this could look like each agent having a network connection to all of the agents that they need to be able to directly communicate with, and each agent would maintain their own respective Python process for communication between each other. pymavswarm was designed without restrictions on network configuration beyond needing a connection to the agents to give users the flexibility to design their network as they choose.

For testing, a centralized network was designed to use RFD900 radios configured using the multi-point network firmware (the datasheet for this is here). Each agent had an RFD900 radio connected to a Pixhawk with messages forwarded to a companion computer (Raspberry Pi 4), and the GCS had a single RFD900 radio. The GCS was responsible for running the MavSwarm process which connected to the radio via a serial connection and for sending commands to the agents. Tests have also been performed with WiFi hardware and MAVLink Router instead of the RFD900 radios to help decrease latency and increase bandwidth; however, these tests also used a centralized network configuration (I am less familiar with the WiFi network configuration, unfortunately).

For a decentralized network (untested on hardware as of now), the hardware and network design could look like this:

To give you some insight on what I know about swarms, I'll explain the simulation project we've developed last year and why I think your project fits in with it. We developed a decentralised swarm simulation in AirSim using PX4s in SITL, each drone using MavSDK-Python to access the PX4 on its own thread as a companion computer. It worked, but the problem is that the communication we implemented is unrealistic for real drones. We wrote the communication messages in files, taking advantage of the fact that they are all running on the same computer. With this approach we managed to make the following features to work: communication between drones and to detect each other and create a swarm, to design a formation, to move in formation in a synchronized manner, to relocate when a drone ceased to be part of the formation, and to avoid obstacles collaboratively (if a drone detected it and another one did not follow behind it). Looking at your library I think it does make sense to incorporate it in our project, as it is a more realistic and formal communication approach, using Mavlink messages.

This sounds like an excellent project :smile: . Something worth noting early is that the tests that we have performed were with ArduCopter. I have done some tests with PX4, but not extensive tests. So there may be some tweaks and additions that need to be made to fully support PX4.

Do you have any initial idea of what could be used to make the rest of the drones receive messages in simulation? Maybe with MaxProxy or some app like that? We have also the ability to perform HITL, in case it is useful to migrate from real to simulation.

This is the big question that will need to be worked through. MAVProxy could be an approach to that. Another idea that I had though was to utilize just MAVLink Router facilitate message distribution (mavp2p could also be used as an alternative). This could possibly look this:

I'm not familiar with the MAVLink Router tool, so I don't know for certain whether this would work, but that could be a place to start. I like your idea of starting with a HITL based approach as well, but I will leave that decision to you.

What are your thoughts? If you think that using the MAVLink Router would be a good place to start, it might be worth dropping a message in the ArduPilot Discord to see what their opinions are on using something like that. If you happen to have questions regarding setting up your development environment, the development process, etc. please feel free to create new discussions!