Mission Planning

[mojodna]

WebODM should facilitate the planning of missions.

Input:

• area(s) to cover (polygon) - if multiple areas, treat as multiple missions?
• angle(s) to take photos
• altitude
• hazards (is this a thing handled by mission planning software?)
• camera specs
• desired overlap
• ...

Output:

• 3D linestring representing flight path
• [flight planning output]
• ...

[Bavar2142]

I know in terms of the phantom 3 the only real hazards it knows are no fly zones around airports. Would this focus on autopilot or more of a you have images of this much of the mission area?

[pierotofy]

It would focus on autopilot (you choose an area to map, the drone takes off and collects the images). I don't think it will support hazard detection (at least initially).

[dakotabenjamin]

I think a good way to handle hazards is to be able to create a geofence, i.e. let the user decide. Most mission planners I've used just do a big circle, but I think it would be more useful to be able to import or draw multi-polygon shapes.

[Bavar2142]

@pierotofy That would be hugely helpful if it worked with phantom 3's as its more by trial and error at the moment. Ie We fly as close to a grid pattern as possible with an hour drive to get there and another back; and then have to spend a few hours processing before we realise the mistake. From memory pix4d generated a report prior to the main stage of processing showing a measure of overlap of the AOI which was rather useful in that regard. How are you planning to get it to talk to the quad copter? android app I assume?

@dakotabenjamin Even just a multi sided polygon would work well from my experience as I could just fence in the area I wanted mapped with everything outside regarded as a no fly zone.

[pierotofy]

DJI has an SDK which works for both iOS and Android. Would probably target Android first, but there are several options.

[Bavar2142]

Ah ok. We use an android tablet for the signal on ours.

[jjmata]

There is an iOS codebase we might be able to start from (I'm trying to coordinate it's FOSS release) but not sure about Android. Is there a timeframe you have in mind here, @pierotofy?

[pierotofy]

@jjmata I would guess probably mid 2017. It might happen sooner if we find more contributors though.

[ns-1m]

I read in one of the pages in OpenDroneMap that somebody is asking for web based mission planner.

Here's a web based mission planner based on Ardupilot.

Skysense Planner Video - Open-source Web Mission Planner for MAVLink Drones http://diydrones.com/profiles/blogs/skysense-planner-video-open-source-web-mission-planner-for

http://www.skysense.co/planner-launch

Skysense Planner’s technology stack is based on Open-Source projects:

Ardupilot: Unmanned aerial vehicle (UAV) platform, able to control autonomous multicopters, fixed-wing aircraft, traditional helicopters, ground rovers and antenna trackers.
Gazebo: Simulator offering the ability to accurately and efficiently simulate populations of robots in complex indoor and outdoor environments.
MAVLink: MAVLink Micro Air Vehicle Communication Protocol.
MAVROS: MAVLink extendable communication node for ROS with proxy for Ground Control Station.
ROS: Robot Operating System, a collection of software libraries and tools developers can use to assist in robot application development.
Robot Web Tools: Integration between ROS and Web applications using Javascript and HTML5.
NASA Web World Wind: 3D virtual globe API for HTML5 and JavaScript.
Dump1090: ADS-B radar receiver (Mode S decoder) specifically designed for RTLSDR devices.

[pierotofy]

Cool! What's with the "request early access" sign-up page on their site? Is there a place to download the code?

[jjmata]

To follow up on my comment about an iOS codebase that could be considered as a starting point for that platform, here is the app currently on the App Store I was referring to above. The team is moving on and interested in having somebody take over development.

It lacks the "lawn mower pattern" workflow and implements the more intricate waypoint + camera pitch one, so upgrading the DJI SDK (at 2.4 a little long in the tooth) and implementing "lawn mower" would be the first priority, IMHO.

[jjmata]

FWIW, it's one of the apps still listed on DJI's developer showcase website - bottom right as of this writing next to Skycatch and PrecisionHawk offerings.

[pierotofy]

It would be really cool if they decided to release the source! It could be used as a good starting point.

[jjmata]

I'm 100% sure that is their intention, and 99% sure that it will happen "soon" (within weeks, not months). I have access to the codebase and have hacked in it myself, so I'm familiar with some of the shortcomings. Happy to help speed things along ...

Do people have any suggestions as to what license (MIT, Apache, GPL, etc.) [1] you would like me to encourage them to use? Who are the most likely developers/candidates to spend some time enhancing the code? I'd love to hear your opinions for sure ...

[1] I was a little surprised about the choice of MPL for WebODM, not due to anything specific to the license (personally, I was at Mozilla when the MPL was concocted and like it ;-) but would love to understand the logic behind that choice.

[pierotofy]

I think either LGPL or MPL would be a good license for the project, but ultimately I think the development team should decide on their own how they want to release it. My focus is going to be on building and consolidating the web interface and api in the short term at the moment, but people could start working on the mission planner side before I get to it (that would actually be great!).

MPL is permissive, encourages code sharing (arguably more so than MIT and Apache2). It also doesn't require a CLA from contributors (unlike Apache2). MPL can be statically linked with proprietary software, which is less of an obstacle compared to LGPL which requires dynamic linking. Overall MPL/[L]GPL are very similar, but MPL seems a tiny bit better.

[pierotofy]

For Android, I also found this project:

• https://github.com/DroidPlanner/Tower
• https://play.google.com/store/apps/details?id=org.droidplanner.android&hl=en

[Bavar2142]

http://developer.dji.com/showcase/4500261e49f807b9ff18833a06660c27/

Does anyone have any experience with this one?

[pierotofy]

@Bavar2142 I don't, have you tried it?

[Bavar2142]

Ill have a word with a local store that sells phantoms as id rather not risk the one we use at work flying off. Ill get back to you once i've had a word with them.

[jjmata]

So @pierotofy is taking a look at the codebase I've been spouting about, and I just invited @mojodna to take a peek as well. Once I hear from them we'll put the wheels in motion to re-license / publish / etc. Assuming it makes sense, of course.

EDIT: by "makes sense" I mean the codebase is a good starting point. The Freeskies team has given me the green light to move this forward in whatever way makes sense.

[jjmata]

Any (early) thoughts @pierotofy and @mojodna? Worth for me to work towards relicensing that codebase and/or put it up somewhere for a wider audience? Based on your feedback I'll bring @dakotabenjamin and @smathermather into the loop to see if it's worthy of inclusion under OpenDroneMap directly or not ...

[pierotofy]

Haven't had time to check it out yet. Will post as soon as I do.

[pierotofy]

I checked out the code base, and here's my take:

1. Code-base is tightly coupled (most functionality is tied together, which is understandable, the software was never designed to be extended or modified later on). Browsing through the code and understanding the inner workings of each part would require a substantial effort,
2. Layout is designed for a landscape experience on iPad screens; things just wouldn't fit (easily) on smaller screens, and the work required to make it iPhone compatible could be challenging.
3. Being coupled with Apple Maps, an Android port would be more difficult.

Perhaps @mojodna can provide another point of view.

Unfortunately I don't think that we will be saving a whole lot of time by adopting this codebase as a starting point. It's a good piece of software, it just doesn't fit our goals.

[jjmata]

I've hacked/extended it for some prototype work myself, and agree with most of your assessment about lack of modularity. One thing I disagree with is the goal of an "Android port" however (number 3 above), in my experience two native experiences would be best if attainable (that is, if we have enough interest to develop two side-by-side apps).

Of course, that could mean divergence and a fractured experience across platforms.

I'm also not surprised/bothered by the landscape/iPad layout. A little bit of work would give us a 5.5" screen version and I prefer tablet frankly for complex workflows (with pen/stylus input in the future!)

The main reason I would be excited to see the "keyframe" approach used by Copilot survive is because it lends itself well to 3D structure mapping via SfM (and that type of mission mapping is way more complicated than surveying ...) I could even imagine a SLAM "interpretation" of the UI when flying in manual mode and simultaneously collecting data [1].

Anyway, let's wait for @mojodna's take as well. I'm thinking I could also just open source outside of the OpenDroneMap umbrella so that we can at least have something available that works already. I'll talk to the FreeSkies team about that option.

[1] OK, getting a little bit ahead of myself there, I know!

[pierotofy]

Perfectly agree with you on open sourcing it!

[mojodna]

I'm going to have a look today and with that, some thoughts ;-)

[mojodna]

The main reason I would be excited to see the "keyframe" approach used by Copilot survive is because it lends itself well to 3D structure mapping via SfM (and that type of mission mapping is way more complicated than surveying ...) I could even imagine a SLAM "interpretation" of the UI when flying in manual mode and simultaneously collecting data [1].

This appears to be the primary value of the codebase (besides being a reference point for real-world use of the DJI SDK and the 3D previews).

I fully endorse open sourcing it, specifically for those reasons, but I agree with @pierotofy that it's probably not the right base to build from, particularly because it doesn't include "table stakes" features like choosing an area and creating a flight plan to cover it (yes, that could be added, but the paradigm seems a bit different).

[jjmata]

OK, spoke to the FreeSkies team and I'm going to put the code up here in the next few days, under MPL 2.0 unless people have thoughts we should consider regarding the choice of license.

[dakotabenjamin]

@smathermather

[smathermather]

MPL 2.0 is fine by me. This is an awesome development.

[pierotofy]

After more research, I think my efforts will go toward development around Tower: https://play.google.com/store/apps/details?id=org.droidplanner.android.beta&hl=en https://github.com/DroidPlanner/Tower

Tentatively, WebODM should provide REST endpoints for authentication, retrieval of missions and upload of images to a new task (this already exists). It will also need to offer a mission planner component. Skysense could be the way to go, but I don't like the fact that they are keeping it in "request early access"... no source code is available yet, so we might have to build this one from scratch (or make a web port of an existing planner).

Once we have a mission planner component, missions can be encoded using this protocol http://qgroundcontrol.org/mavlink/waypoint_protocol which can be read and integrate seamlessly in Tower (and other GCS software for the matter).

At some point I'll ask the Tower developers if they are interested in integrating WebODM, or if we should just make a fork.

The reasons for choosing this are because:

• A lot of people have been developing Tower for quite some time, so we have the chance to tap into an existing community of users and developers
• The ecosystem around Tower is open source
• Their survey option for waypoints creation seems fairly well designed

As an owner of a DJI I'm sadden that I won't be able to use my drone directly for this. But supporting DJI would require more resources and at this point in time I wouldn't have time to rewrite from scratch a mission planner. We could support both obviously, and I look forward to hearing people willing to contribute a DJI / iPhone mobile app.

[pierotofy]

In regard to Skysense, I found this: https://github.com/auturgy/planner

Seems to be missing a survey (grid) waypoints mode. The code might be out of date also, and maybe there's a new version out there.

[pierotofy]

Another quick update: there might be a way to simply let WebODM be a storage for mission plans and let other mission control software handle the rest. For example, qgroundcontrol uses simple JSON files to define their missions: https://github.com/mavlink/qgroundcontrol/blob/master/src/MissionManager/MissionController.cc

Perhaps WebODM could simply expose an API to allow missions to be saved/loaded and then develop a simple plugin for qgroundcontrol.

[dakotabenjamin]

I like that idea.

For other planning software (which may or may not be plugin-able but it's worth knowing): SenseFly eMotion: XML Mission Planner: .waypoints plain text (below)

QGC WPL 110
0   1   0   16  0   0   0   0   41.123809   -81.527932  237.557778  1
1   0   0   22  20.000000   0.000000    0.000000    0.000000    0.000000    0.000000    30.000000   1
2   0   0   16  0.000000    0.000000    0.000000    0.000000    41.128597   -81.534769  120.000000  1
3   0   0   206 25.753845   0.000000    0.000000    0.000000    0.000000    0.000000    0.000000    1
4   0   0   16  0.000000    0.000000    0.000000    0.000000    41.128884   -81.532854  120.000000  1
5   0   0   206 0.000000    0.000000    0.000000    0.000000    0.000000    0.000000    0.000000    1   

[smathermather]

I like this idea too. I wonder how it fits in with Pacific Drone Imagery Dashboard and similar projects. Any thoughts @cgiovando?

[mojodna]

Litchi also works with CSV/TSV plain text (similar to Mission Planner).

[aaelsay2]

I have been working on web based 3D flight planning for construction visual data collection specifically. According to my experience with mission planning in 3D environment I would say that the major challenge I am facing is the 3D representation of the planning environment. My web based mission planner utilizes webgl scene to create and manipulate waypoints. The waypoints data is then exported to json format and sent to a developed mobile application built over DJI SDK. I am using as-planned 3D architectural models of buildings with Geo-referenced survey points for creating missions. However, this only represents the building and disregards the surrounding 3D environment (ex. nearby buildings and other obstacles). I think it is important to use a comprehensive 3D map API which I would say Google Earth API but unfortunately, the web based Google Earth API is deprecated for a while now. I really hope they return it back for saving many developers' lives. Now, I am looking for a reliable web-based 3D map API. I am open to any suggestions and discussion.

[pierotofy]

Have you checked out Cesium? http://cesiumjs.org/

[aaelsay2]

Yes I tried using Cesium, the 3D map is limited to some cities and actually there are lots of issues you have to fix to make Cesium work.

[pierotofy]

Some work has been started by @pepperlk at https://github.com/pepperlk/mission-planner to create a mission planner interface.

[chrowe]

It would be nice to have some guides to help people get imagery from their drone to ODM via a mission planner.

I just got a 3dr Solo and will be trying to figure out the best solution. Does it make sense to create a wiki page and start documenting the current options?

[pierotofy]

@chrowe I've added a new wiki page here https://github.com/OpenDroneMap/WebODM/wiki/Mission-Planner-Options. Any user should be able to edit it.

[AnthonyCoughlin]

@pierotofy

I fly Autel Robotics and they recently released their SDK. I don't know much about it as far as the code, but if its easy for you guys to manipulate I'll gladly test.

[pierotofy]

@dakotabenjamin do you know of a good resource for the specification of the mavlink waypoint format (.waypoints)?

[dakotabenjamin]

All I have is this link: http://www.qgroundcontrol.org/mavlink/waypoint_protocol at the bottom

[juvinski]

Hi, I have some experience with Mavlink, how can I help?

[pierotofy]

@dakotabenjamin mm yes I had found that link also. If anyone can find a link that explains the possible values for each parameter:

QGC WPL <VERSION>
<INDEX> <CURRENT WP> <COORD FRAME> <COMMAND> <PARAM1> <PARAM2> <PARAM3> <PARAM4> <PARAM5/X/LONGITUDE> <PARAM6/Y/LATITUDE> <PARAM7/Z/ALTITUDE> <AUTOCONTINUE>

It would facilitate the task.

@juvinski are you familiar with the waypoints protocol?

[juvinski]

@pierotofy , yes. In true the paramX is relative to Command you are using. For instance, CMD Takeoff (#22) Param1 is desired pitch angle for takeoff while for MAV_CMD_NAV_WAYPOINT(#16) is the the hold time - https://pixhawk.ethz.ch/mavlink/ - Section MAV_CMD there is all commands and parameters. How can I be helpfull ?

[kikislater]

Mission planner needs *.waypoint file which look like you speak above but qgroundcontrol needs a json file.

If this could help : actually in mission planning in mission planner there is a terrain following option. Aircraft will follow relief but it's not a good idea if you have hills or mountain with huge slope. To remain to this problem in ebee with emotion2, they bring the max altitude of each line of cross grid. Mission planner and emotion2 use srtm 90 to get elevation (mission planner use srtm 30 but only in US) but sometimes is not sufficient : due to smoothing on srtm, I got 2 times difference with 200m which could cause crash. Srtm30 is possible to use all over the world. For plane, looking for meteo is a must and allow autotakeoff and landing in front of the wind. You could watch a such thing in a custom mission planner here : http://littlesmartthings.com/wp-content/uploads/2015/05/MP-tutorial2.pdf Last thing is the way to fly plan along a line to follow a road or a pipeline or a river or ...

This is just my feedback about flight plan !

[arasharchor]

hi, may I ask why mission planner feature has not been added? Is there any specific problem which no one has been able to solve it?