NameError: name 'wgs84' is not defined in 1c-make-pix4d.py

ArunLukeDsouza commented 3 years ago

Hi again! I've been trying to use this project on a custom aerial dataset (crop_field). I tried running 1c-make-pix4d.py with

python3 1c-make-pix4d.py crop_field

and it yielded

Traceback (most recent call last):
  File "/Users/ImageAnalysis/scripts/1c-make-pix4d.py", line 26, in <module>
    pose.make_pix4d(image_dir, args.force_altitude, args.force_heading, args.yaw_from_groundtrack)
  File "/Users/ImageAnalysis/scripts/lib/pose.py", line 229, in make_pix4d
    wgs84.geo_inverse( cur[1], cur[2], next[1], next[2] )
NameError: name 'wgs84' is not defined

Am I missing a package or dependency? I've tried looking for a workaround but didn't come across anything related

clolsonus commented 3 years ago

Oh, you have entered a tricky part of the code! From a quick glance that code path might need some work, the required import for that code path to work is missing.

Background: with many systems you can extract the roll/pitch/yaw of the camera when the image was taken (the camera "pose"). This is typically the case with DJI and Sentera imagery. When this information is not available in the image, often it can be derived from the flight data if that is saved and correlated with the image timing. When none of that is possible, then the system can assume the camera was mostly pointed straight down and try to estimate the yaw angle of the camera from the ground track, which is the code path you found (but only something that I've had to do in very old data sets.)

So maybe it would be better to take a step back and talk about what camera and drone system you are using, how your camera is mounted to the drone (fixed mount, gimbal?) Does the camera heading track the drone heading? Is this a fixed wing drone? a multicopter?

The Sentera mission planner has the drone fly the entire mission on a fixed heading (90 degrees) and that can be convenient if we know that ahead of time. The camera pose estimate doesn't have to be perfect, but the system depends on it being somewhat in the ball park of reality.

Also, if you are able to share the data set, I could look at it here and possibly offer more insight and direction (or possibly not ...) but I'm willing to take a look. You can send big data sets directly to me using filemail (I have a subscription): https://curtolson.filemail.com/

Essentially the system needs to have an initial guess for the cameras location and attitude for each picture taken, that is what the pix4d.csv file provides and that is what said code is trying to create/estimate for you from information available.

ArunLukeDsouza commented 3 years ago

@clolsonus Thank you for the spontaneous reply!

I'm currently working with Sensefly Datasets (specifically Crop Field (RGB)) which can be found here collected by an eBee SQ drone carrying a Parrot Sequoia camera (I presume you'll find the other details in the link as well).

I've also had another minor doubt, while running 99-new-camera.py it yielded Cannot autodetect calibrated focal length, please specify a ccd-width (which I believe is the focal length), a quick google search gave me 3.98mm as the value. However, the values in EXIF gave me 4.88mm, which value do I choose for the ccd-width parameter?

clolsonus commented 3 years ago

@ArunLukeDsouza Oh, in that case we can just trivially cheat and grab the camera calibration numbers out of the pix4d quality report.

clolsonus commented 3 years ago

I created a camera config for the Parrot Sequoia and modified the exif pose extraction code to find the Parrot camera post values (which are conveniently included in their metadata.) Try doing a git pull, then completely remove the ImageAnalysis working directory that was created inside the image folder, and try running process.py /path/to/image/folder again from scratch.

clolsonus commented 3 years ago

And sadly, parrot is completely lying about it's camera yaw angle in the image metadata (unless I'm completely missing something.) The camera is reporint yaw angles of about -50 to -70 degrees (290 to 310-ish?) But from the shadows you can see the aircraft and camera have changed orientation by up to 180 degrees without the matching change in yaw angle in the meta data. That's a bummer. :-( That will cause my tool chain problems later when it tries to build an initial estimate of the 3d feature locations before giving them over to the optimzer to refine.

ArunLukeDsouza commented 3 years ago

@clolsonus Thanks a million for the fix! I was able to get past that! However while running python3 1b-set-camera-config.py crop_field it yielded

Camera: ../cameras/Parrot_Sequoia.json
Unknown child type: mount <class 'props.PropertyNode'>

Not much of an issue I guess?

clolsonus commented 3 years ago

You do need to install the aura-props package from here:

https://github.com/RiceCreekUAS/aura-props/tree/master/python

(You don't need the C++ api, just the python side.)

Curt.

On Wed, Feb 3, 2021 at 7:48 AM ArunLukeDsouza notifications@github.com wrote:

@clolsonus https://github.com/clolsonus Thanks a million for the fix! I was able to get past that! However while running python3 1b-set-camera-config.py crop_field it yielded

Camera: ../cameras/Parrot_Sequoia.json Unknown child type: mount <class 'props.PropertyNode'>

Not much of an issue I guess?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/UASLab/ImageAnalysis/issues/11#issuecomment-772518308, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACXYYDITWP6RDS3BCFU72KDS5FHZPANCNFSM4W6DG6NA .

-- Curtis Olson University of Minnesota, Aerospace Engineering and Mechanics, UAS Lab

ArunLukeDsouza commented 3 years ago

During the process of feature detection and marching, I executed python3 3a-matching.py crop_field and it yielded

Loading keypoint (pair) matches:
  0%|                                                                                          | 0/356 [00:00<?, ?it/s]crop_field/ImageAnalysis/meta/IMG_160729_071349_0000_RGB.match:
  matches load error: <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'crop_field/ImageAnalysis/meta/IMG_160729_071349_0000_RGB.match'
crop_field/ImageAnalysis/meta/IMG_160729_071351_0001_RGB.match:
  matches load error: <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'crop_field/ImageAnalysis/meta/IMG_160729_071351_0001_RGB.match'
crop_field/ImageAnalysis/meta/IMG_160729_071353_0002_RGB.match:
  matches load error: <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'crop_field/ImageAnalysis/meta/IMG_160729_071353_0002_RGB.match'

and this error seems to go on until the last image. However on the last image it works upto a 100% but yields another error

crop_field/ImageAnalysis/meta/IMG_160729_073121_0355_RGB.match:
  matches load error: <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'crop_field/ImageAnalysis/meta/IMG_160729_073121_0355_RGB.match'
100%|█████████████████████████████████████████████████████████████████████████████| 356/356 [00:00<00:00, 29139.62it/s]
NED reference location: [46.52177338208877, 6.552037014359831, 0.0]
Initializing the SRTM interpolator
SRTM: loading DEM tiles
SRTM: downloading: https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Eurasia/N46E006.hgt.zip
('/var/tmp/N46E006.hgt.zip', <http.client.HTTPMessage object at 0x7fcbf0130310>)
SRTM: parsing .hgt file: /var/tmp/N46E006.hgt.zip
SRTM: constructing LLA interpolator
SRTM: constructing NED area interpolator
Traceback (most recent call last):
  File "/ImageAnalysis/scripts/3a-matching.py", line 102, in <module>
    surface.update_srtm_elevations(proj)
NameError: name 'surface' is not defined

Have I missed out on the feature extraction part?

ArunLukeDsouza commented 3 years ago

You do need to install the aura-props package from here: https://github.com/RiceCreekUAS/aura-props/tree/master/python (You don't need the C++ api, just the python side.) Curt. …

Well I already had the Part 1 aura-props package installed after looking at the previous issues, I went ahead and did a clean install again still shows Unknown child type: mount <class 'props.PropertyNode'>

clolsonus commented 3 years ago

Ok, there is an issue there, the message is coming from the aura-props code when overlaying one tree on top of another. Likely the whole "mount" subtree is lost which means the project won't default to the proper camera mounting (straight down.) I don't fully understand why I haven't seen this before and why this case isn't accounted for in the aura-props code. I need to dig deeper and try to understand and refresh my memory on that, but I probably don't have time today. For now I woudl recommend adding the following snippet to your ImageAnalysis/config.json file right after camera make, model lines (and before width):

    "mount": {
        "pitch_deg": -90.0,
        "roll_deg": 0.0,
        "yaw_deg": 0.0
    },

The sequoia meta data for roll, pitch, and yaw is for the aircraft (and yaw is completely wrong all the time.) So you need to trigger the code --yaw-from-flighttrack and tell the system the mount offset of the camera when creating the pix4d.csv file.

clolsonus commented 3 years ago

I spent some more time working on this sequoia data set and have a few more follow up comments.

The initial pose estimate in the image metadata is extremely inaccurate which causes my code a lot of trouble later when trying to optimize and fit all the features/poses together.
It's a good demo set overall for pix4d which handles freeform data a lot better than my tools.
These tools here are designed to look for needles in a haystack, not to build 3d models of structures and towns.

If you are interested here is a different data set (captured with Phantom 4 pro) from a very low altitude over a mature corn field. This is a data set that is very difficult for pix4d to handle and allows you to even look down through the corn to the bare ground underneath. For whatever it's worth if you don't mind a 5Gb download! You can view this with the scripts/explorer.py tool that is part of this project: https://drive.google.com/file/d/1r1uV5lutFEpqabBxwlHO-Eg2oDsm_gKp/view?usp=sharing

ArunLukeDsouza commented 3 years ago

@clolsonus I think you're right about the issues with the sensefly dataset. So I decided to abandon the sensefly dataset and tried reproducing results with your datasets, both the datasets (Avon Park & Corn Field) work perfectly with the explorer. However when I deleted the configuration files that came with them keeping only the images, to see if I could reproduce your results from scratch, I faced this issue again :(

During the process of feature detection and marching, I executed python3 3a-matching.py crop_field and it yielded

Loading keypoint (pair) matches:
  0%|                                                                                          | 0/356 [00:00<?, ?it/s]crop_field/ImageAnalysis/meta/IMG_160729_071349_0000_RGB.match:
  matches load error: <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'crop_field/ImageAnalysis/meta/IMG_160729_071349_0000_RGB.match'
crop_field/ImageAnalysis/meta/IMG_160729_071351_0001_RGB.match:
  matches load error: <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'crop_field/ImageAnalysis/meta/IMG_160729_071351_0001_RGB.match'
crop_field/ImageAnalysis/meta/IMG_160729_071353_0002_RGB.match:
  matches load error: <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'crop_field/ImageAnalysis/meta/IMG_160729_071353_0002_RGB.match'

and this error seems to go on until the last image. However on the last image it works upto a 100% but yields another error

crop_field/ImageAnalysis/meta/IMG_160729_073121_0355_RGB.match:
  matches load error: <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'crop_field/ImageAnalysis/meta/IMG_160729_073121_0355_RGB.match'
100%|█████████████████████████████████████████████████████████████████████████████| 356/356 [00:00<00:00, 29139.62it/s]
NED reference location: [46.52177338208877, 6.552037014359831, 0.0]
Initializing the SRTM interpolator
SRTM: loading DEM tiles
SRTM: downloading: https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Eurasia/N46E006.hgt.zip
('/var/tmp/N46E006.hgt.zip', <http.client.HTTPMessage object at 0x7fcbf0130310>)
SRTM: parsing .hgt file: /var/tmp/N46E006.hgt.zip
SRTM: constructing LLA interpolator
SRTM: constructing NED area interpolator
Traceback (most recent call last):
  File "/ImageAnalysis/scripts/3a-matching.py", line 102, in <module>
    surface.update_srtm_elevations(proj)
NameError: name 'surface' is not defined

Have I missed out on the feature extraction part?

and this one too,

@clolsonus Thanks a million for the fix! I was able to get past that! However while running python3 1b-set-camera-config.py crop_field it yielded
Camera: ../cameras/Parrot_Sequoia.json
Unknown child type: mount <class 'props.PropertyNode'> 
Not much of an issue I guess?

clolsonus commented 3 years ago

I apologize, I've kind of lost sync with the discussion. You said you had switched to testing with the dataset I provided, and are seeing the same errors, yet you are posting errors from the sequoia data set. So I'm confused ... sorry about that.

ArunLukeDsouza commented 3 years ago

My bad I should’ve been more clear about it. I meant the DJI dataset yields similar errors to the Sensefly dataset when I deleted the configuration files that came with them keeping only the images, to see if I could reproduce your results from scratch

When I run

python3 1b-set-camera-config.py phantom-23-20190731

It yields

Camera: ../cameras/DJI_FC6310S.json
Unknown child type: mount <class 'props.PropertyNode'>

And when I run

python3 3a-matching.py phantom-23-20190731

It yields this massive error

Loading keypoint (pair) matches:
  0%|                                                   | 0/648 [00:00<?, ?it/s]phantom-23-20190731/ImageAnalysis/meta/DJI_1012.match:
  matches load error: <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'phantom-23-20190731/ImageAnalysis/meta/DJI_1012.match'
phantom-23-20190731/ImageAnalysis/meta/DJI_1013.match:
  matches load error: <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'phantom-23-20190731/ImageAnalysis/meta/DJI_1013.match'

Which goes on until the last image

phantom-23-20190731/ImageAnalysis/meta/DJI_1659.match:
  matches load error: <class 'FileNotFoundError'>: [Errno 2] No such file or directory: 'phantom-23-20190731/ImageAnalysis/meta/DJI_1659.match'
100%|██████████████████████████████████████| 648/648 [00:00<00:00, 19809.98it/s]
NED reference location: [43.92207759004631, -95.90702347586672, 0.0]
Initializing the SRTM interpolator
SRTM: loading DEM tiles
SRTM: parsing .hgt file: /var/tmp/N43W096.hgt.zip
SRTM: constructing LLA interpolator
SRTM: constructing NED area interpolator
Traceback (most recent call last):
  File "/Users/arun/Downloads/AERO2ASTRO/Project GIS/ImageAnalysis/scripts/3a-matching.py", line 102, in <module>
    surface.update_srtm_elevations(proj)
NameError: name 'surface' is not defined

NorthStarUAS / ImageAnalysis

NameError: name 'wgs84' is not defined in 1c-make-pix4d.py #11