Mavic 3M multispectral alignment issues

ehallein commented 3 months ago

How did you install ODM? (Docker, installer, natively, ...)?

Docker

What is the problem?

Multispectral bands are not aligned in orthophoto

What should be the expected behavior? If this is a feature request, please describe in detail the changes you think should be made to the code, citing files and lines where changes should be made, if possible.

aligned bands

How can we reproduce this? What steps did you do to trigger the problem? If this is an issue with processing a dataset, YOU MUST include a copy of your dataset AND task output log, uploaded on Google Drive or Dropbox (otherwise we cannot reproduce this).

console.txt output and images https://drive.google.com/file/d/1p2bUUqzd-TxL-kPQttbjuv-PtB49WtzS/view?usp=sharing

create project in webODM (using nvidia GPU) and run using multispectral option on TIFFs

possibly related to https://www.reddit.com/r/UAVmapping/comments/1cacp09/processing_multispectral_dji_mavic_3m_data_with/ "First, locate the fields 'Relative Optical Center X' and 'Relative Optical Center Y' in the EXIF information of each band's photograph. These values represent the positional offset of each camera band relative to the NIR band camera (in pixels). By setting the compensation values (offset) for each band's photograph (excluding the NIR band) in software like ENVI, you can correctly combine the images. The specific method for setting the compensation values for each band will need to be referenced from the manual of ENVI or similar software."

pierotofy commented 3 months ago

https://github.com/OpenDroneMap/ODM/pull/1740

ehallein commented 3 months ago

https://github.com/OpenDroneMap/ODM/pull/1740

That's seems to be related to differences is the RGB vs multispectral lenses? There no RGB being used in my case, just the multispectral bands.

smathermather commented 3 months ago

Thanks for opening the issue.

That's seems to be related to differences is the RGB vs multispectral lenses? There no RGB being used in my case, just the multispectral bands.

The problem is sufficiently related to be considered the same: either we choose a multispectral band to align to, as per @mariotro's example here: https://gitlab.com/Yario/image_registration_dji_mavic_3m/-/blob/main/coregistration_mavic_3m_images_with_CV2.py?ref_type=heads or we align to the RGB per @lewispb's example here: https://gist.github.com/lewispb/6c7dd6022ba3266cfd4f1310d4d51f5f

In the second scenario, the exif data for that alignment is extant in the multispectral bands, so it wouldn't require RGB upload with the MS bands, so it should be independent of how exactly the data are processed.

pierotofy commented 3 months ago

This should be fixed with https://github.com/OpenDroneMap/ODM/pull/1771

The problem was multifaceted:

A fault in how the homography matrices were computed on images larger than 1280 pixels
The RGB lens model being too different than the multispectral ones

Both should be addressed but some testing will be needed.

ehallein commented 3 months ago

Excellent. Happy to help test.

ehallein commented 3 months ago

This should be fixed with #1771

The problem was multifaceted:

A fault in how the homography matrices were computed on images larger than 1280 pixels

The RGB lens model being too different than the multispectral ones

Both should be addressed but some testing will be needed.

looks much better with now with the new code. although still a slight offset with redband compared to G,NIR and Red edge which all line up.

pierotofy commented 3 months ago

@ehallein could you post your task processing log?

ehallein commented 3 months ago

@pierotofy here you go console.txt

pierotofy commented 3 months ago

Thanks!

I don't think it might matters in this case, but always re-run from the dataset stage between version updates.

Mm, not sure why the red band ended up off. A link to the dataset (or better, a smaller subset) would help (the link on google no longer works?).

ehallein commented 3 months ago

Thanks!

I don't think it might matters in this case, but always re-run from the dataset stage between version updates.

Mm, not sure why the red band ended up off. A link to the dataset (or better, a smaller subset) would help (the link on google no longer works?).

here are TIFFs https://drive.google.com/file/d/1OEulM3PKqwOrRDX9oCg54osxpzyJNoTg/view?usp=sharing The restart was just a redo to test. I've run a few times, including both mac (m1 pro) and windows with same issue. I'm trying some other datasets now.

ehallein commented 3 months ago

Here's image from another dataset. Still out. Only red band. But it's pretty close, so maybe it's just the error on the lens data.