PlantPhenoHack2017 / HackTopics

Use the issue tracker of this repo to suggest topics for hacking.
0 stars 0 forks source link

UAV/Drone quality control #1

Open westerman opened 7 years ago

westerman commented 7 years ago

I would like to the explore the idea of post-flight UAV quality control. See the attached PDF. It may be possible that such a program has either (1) already been written - although I do not know of one -- or (2) more likely simply rendered obsolete by during-flight mission control programs. It seems to be more common that UAVs will report to a ground station and either abort the mission if something goes wrong or the pilot can stop the mission if some amiss is detected. On the other hand this does rely on a pilot who has good mission control software and is aware of what is happening.

Perhaps last year I just had a pilot who wasn't very good but he managed, on various flights, to:

1) Fly too low

2) Not fly enough passes.

3) Take manual control of the drone and fly erratic patterns - granted he did have to avoid a power line. Also he said that on a couple of occassions the plane was flying erratically.

4) Set the shutter speed to 3 seconds instead of 1 second thus taking 1/3 the number of images

5) Not turn on the camera -- or something -- I got a bunch of blank images.

6) Fly over the wrong field

Now he is a nice enough of a guy but, still, all of the above makes me want to have a 'stick' to hit him over the head with. Especially since his process was 'fly, go home, take off images from camera, put them a USB stick, bring it back in to the office for a hand-off and let Rick look at them' with a consequent multi-day delay between the flights and my 'oh what has he done' moments. We lost most of the growing season.

So my idea of a quick post-flight quality control check program. Some way for him to plug in the camera or download images to a computer and within a couple of minutes get a 'yes, good job' or 'oops, a mistake has been made' report. Some common method between him and me to verify his flights. Yes, computational solutions are not the ultimate solution to process-control problems but they can help. So if we can develop a QC program and/or figure out other solutions then I would appreciate it.

Proposal for Hackathon project.pdf

westerman commented 7 years ago

I can provide around 7 flights of various quality for testing. Don't know if I will upload them via GitHub but at the very least will post some URLs linking to them. However I might not get this done for several days.

The camera we used is a 'red edge'. We used a couple of different fixed wing UAVs.

westerman commented 7 years ago

During the 1/25/17 hackathon planning session Tyson Swetnam added some comments into my document. I put them here because they are of interest. Not all refer to immediate post-flight QC but rather to further downstream processing. That is also a topic which we may wish to address.

Tyson: I have experience using several DJI centric UAV programs: Altizure: https://www.altizure.com/features#title/2, DJIGo: http://www.dji.com/goapp, and DroneDeply: https://www.dronedeploy.com/. These softwares typically use the EXIF tag from the DJI cameras to report the camera settings - images report their focal length as well as the GPS location of the unit at time of capture. These services all have pay-to-play image processing and model creation levels, if you’re willing to use these.

Do-it-yourself UAVs, like the Firefly6 https://www.birdseyeview.aero have real potential - but their software is years behind the groups like DJI. Lack of integration between the UAV brain (Pixhawk) and the various cameras/sensors was a real hardship for me in my experience.

Real-time image checking is available using commodity hardware like DJI - http://shotkit.com/dji-phantom-4-review-for-photographers/

For multi-spectral cameras, my only experience is with SenseFly Ebee https://www.sensefly.com/drones/ebee.html

The Ebee does have real time image quality sensing - a cloud came over our area of interest and changed the insolation of the surface. The Ebee detected this change and rejected its mission plan and came back immediately.

Post-processing (not real-time) of imagery or video can be done in VisualSFM (open-source), Agisoft Photoscan ($$$), or Pix4d ($$$$) to generate 3D meshes and point clouds. Once images have a Bundle *.out file they can be reprojected in orthoprojection and mosaicked as 2D images.

VSFM: http://ccwu.me/vsfm/ CloudCompare: http://www.danielgm.net/cc/

Rick: Thanks for all of the above. I have used Photoscand and Pix4D but not VisualSFM. Will have to check out the Ebee