Does ODM support Tif format

[xialang2012]

I often use the multi-spectral camera to capture images in the experiment filed. So, I wonder if I can use Tif format files with multi-band, e.g. 10 bands in one Tif image, as input source images after some modifications. Is it a heavy work if I do this modifications? Thank you.

[smathermather]

Pull requests welcome. TBH, I don't know what the implications are of multi-band (>3) tiffs... . @paulinus -- how wedded is the OpenSfM portion of the toolchain to the idea that the images are 3-band (let alone particular image formats)?

[xialang2012]

@smathermather it means there are several channels images in one tiff file.

[paulinus]

The SfM part of the pipeline only needs grayscale images. So you could convert the TIFs to RGB or grayscale jpg and run it. Which conversion to do from 10 channels to RGB or grayscale will depend on what those channels are, but it will not be very critical for SfM.

Stereo and meshing will also work fine just by using the converted images.

Texturing and orthophoto generation are more problematic, since they would need to deal with the 10 channels directly in order to produce a 10 channels output. I ignore how much work is required to modify these parts.

[smathermather]

Perfect. Thanks @paulinus.

[xialang2012]

@paulinus It seems only the orthophoto is problematic. The work mainly focus on change the RGB to one channel, and revise the CV functions which used in the OdmOrthoPhoto.cpp.

[ivopavlik]

I would highly appreciate tiff support. It would bring many advantages, mainly for multispectral sensors, even it is challenging. Precision is lost with conversion to JPEG, since tiffs can use other data types (up to 16 bit uints, floating numbers), since JPEGs use only 8bit data values.

[dakotabenjamin]

I think mvs-texturing can support tiff inputs so I think its just a matter of doing the conversion to greyscale for opensfm while keeping the original images registered to the point cloud (nvm)

edit: there's a branch for n-channel support but I don't think its implemented yet: https://github.com/nmoehrle/mvs-texturing/commits/n-channel-images

[xialang2012]

Unfortunately, the mvs-texturing does not support the n-channel well. I have do some work on this branch, but due to the problem about the https://github.com/nmoehrle/mvs-texturing/issues/49 which is very difficult to solve. In fact, the https://github.com/nmoehrle/mvs-texturing/commits/n-channel-images does not run well on it's current form.

[LucasMM86]

Hello, i was working with the camera Parrot Sequoia. It has 4 bands. The output of this camera is one tiff image for each band (4 images for each shot), 16 bits resolution. I wanted to do NDVI orthophoto (only 2 bands RED and NIR are needed). I did this steps:

1. make jpg images with RED and NIR channels, with a simple python routine. In this step I had to reduce the resolution 16bits -> 8 bits :(
2. run ODM with the jpg images.
3. calculate NDVI with the ortophoto, with a python routine.

The results are OK as you can see, but i guess it is better to calculate NDVI with the 16bits channels.

is it possible to run the texturing step several times in order to ge one orthophoto for each channel?

[dakotabenjamin]

Very cool. Can you share the scripts you used? We also have a Sequoia.

Texturing uses the generated .nvm file which registers specific images to the point cloud. You would have to run ODM completely with each channel separately.

I've never tried this but maybe you could spoof the image file names in the nvm file. I would recommend compiling mvs-texturing on its own and running it that way, probably easier than using ODM for that.

[dakotabenjamin]

Thanks for the update @xialang2012 I'm excited to see how that turns out.

[Fi156]

@LucasMM86 : Please be aware of the automatic gamma correction, which gets applied by the tiff>jpg conversion. IMO this will have a bigger impact on your data than 16 >> 8 bit conversion!

[LucasMM86]

Thanks for the advice @Fi156 . This are the scripts i used. TiffJPG_python.zip

[hblanken]

@LucasMM86 very much like your NDVI creation. Would the quality of the output be any better if you reverse the workflow? I.e. 1. calculate NDVI on each NIR and RED Tiff image pair via python and save as new tiff image 2. convert resulting ndvi image to jpg via python 3. upload NDVI image to ODM and stitch into ortho. I read in #182 that feature extraction might be more difficult, but just curious if anyone has had success and this would result in any better output? Clearly tiff support would be even better.

[xialang2012]

Since the MVS-texturing does not work well for n-channel, the correct way to achieve it is as follow

1.convert to 8bits, and get meshing file 2.use the ODM texturing module to replace the MVS-texturing, and do a small revision of the ODM texturing module to make it correct to work with 16 bit, texture each channel 3.get orthophoto for each channel 4.calculate NDVI

The problem is the ODM texturing module lacks some functons, e.g. blending, hole filled, so the orthophoto obtains by using the ODM texturing module may not as good as MVS-texturing does.

I have completed this last year, but I can not find it now.

[LucasMM86]

@hblanken : I tried to run ODM with NDVI images and the result where not very good. I have not the images here but i remember that the orthophoto had bad quality.

@xialang2012 : I am trying to go throw the 4 steps. I have trouble on step 2. I put the .tif 16 bit images in the images_resize folder and try to run with "--rerun-from mvs_texturing" option. I get the errror "Not enogh supported images in /home/lucas/Documents/Leones/images_resize" ¿should I change the code in order to avoid this error message? ¿or should I use another format for the images?

[smathermather]

Running ODM on any index (ratio data) directly is unlikely to give a good result. The entire purpose of NDVI and other indexes is to remove the noise of features and focus on spectral differences, but for matching, features are what we want.

[ivopavlik]

Hello. I tried to convert my tiffs to 3 band byte jpegs. Then I processed each band separately and then I built virtual RGB raster from RGB separate bands. Unfortunately, bands are shifted each other quite significantly as you can see below. Any clues how to optimize process would be appreciated.

[LucasMM86]

@ivopavlik why do you process each band separately? why do you not process the JPGs with the 3 bands together? The result you are showing has orthophotos of different bands?

[ivopavlik]

Pictures were taken with Micasense RedEdge multispectral sensor. It has separate lenses for each band shifted few cms from each other. It takes picture (by the way tif) for each band separately, where is also some shift and it is not possible to merge it to single image.

[LucasMM86]

I made processing with images of a parrot sequoia (has 4 different lens, 4 bands, tiff images). The images had the same shift problem like your images. You can see the result in my first comment of this issue. The result has not a good resolution due to the shift problem, but maybe is OK for NDVI applications... I made JPG images with NIR and RED bands in each image, so I had to run ODM only once.

[mirosvanek]

I also have the same multispectral camera. If I merge images before ODM processing, each band has significant shift, so it is unusable. Processing each band separately is good workeround, unfortunaltelly shift is also present in result as @ivopavlik wrote. Is there any way how to align resulting mosaic?

[smathermather]

Hi @mirosvanek -- there's no tool in ODM for doing that alignment, but one would be very useful. Can you open a ticket for one as a future feature request?

[ivopavlik]

I opened it (https://github.com/OpenDroneMap/OpenDroneMap/issues/500).

Thanks.

[hblanken]

Hi all - I did some further investigation on how to transfer sequoia images into stitched ndvi orthomaps. I find a sample set: https://support.pix4d.com/hc/en-us/articles/211923706-Example-Datasets-Available-for-Download-Sequoia-Multispectral#gsc.tab=0 I take a script similar to @LucasMM86 used - I used these with the red images and NIR: https://gist.github.com/celoyd/7443646, then I convert them to jpg and overlay with palette to show ndvi index. and finally I stitch them. Result example is below. Remarks: a) Does this method result in an accurate display of ndvi index? Not sure yet - how can I test? b) I cannot stitch in ODM unfortunately as the parrot sequoia images do not contain exif gps information. What do you think about this method? And ideally, if is accurate, I would love to all this with ODM :)

[dakotabenjamin]

I tested NDVI using the Raster Calculator in QGIS on my CIR images (3 band NIR-G-B)

I can post some multispec images with GPS for us to test. I too want this in the pipeline, I am aiming for the v0.4 release.

[dakotabenjamin]

Also, we now have an NDVI script in ODM: https://github.com/OpenDroneMap/OpenDroneMap/tree/master/contrib/ndvi

[LucasMM86]

I think the correct way to do multispectral things is something about what @xialang2012 said: "Since the MVS-texturing does not work well for n-channel, the correct way to achieve it is as follow

1.convert to 8bits, and get meshing file 2.use the ODM texturing module to replace the MVS-texturing, and do a small revision of the ODM texturing module to make it correct to work with 16 bit, texture each channel 3.get orthophoto for each channel 4.calculate NDVI

The problem is the ODM texturing module lacks some functons, e.g. blending, hole filled, so the orthophoto obtains by using the ODM texturing module may not as good as MVS-texturing does.

I have completed this last year, but I can not find it now. " In this way we obtain at point 3 a 16 bit orthophoto for each channel, so we can do NDVI and other index with Qgis or another soft.

I want to try this but I am not sure if my programming skills are good enought.

Is someone thinking or trying another strategy?

[dakotabenjamin]

I threw up a small subset of multispec images taken with the Parrot Sequioa. These are georeferenced. https://github.com/OpenDroneMap/odm_data_garfield_msp

Note that I can't get them to work well in ODM, but they work in Photoscan no problem. I'm looking into it.

[ivopavlik]

In case of using Parrot Sequioa and similar multispectral sensors and computing NDVI, the most important thing is to align resulting orthomosaics since each band has some shift from the other one. Merging them by their georeferenced position (e.g. with QGIS, or gdal tools) has effect, that e.g. NIR pixels has vegetation and RED pixels are on bare soil. Therefore, NDVI will be inaccurate.

[markwbrown]

In an attempt to get around the orthomosaics being shifted from the separate lenses, I aligned and merged the bands into two band tiffs (IR and Red) and THEN processed in photoscan. The result left some rather abrupt stitch lines in the NDVI

[agustinbus]

Hi all, i have a Micasense RedEdge camera, with 5 lens, that take 5 images every shoot. I was analyzing the camera parameters of pix4d, and apparently (my opinion) it is necessary to make corrections for the position of the lenses. I deduced this, since in the list of parameters for each band in pix4d, there are two values: Transl X [mm] and Transl Y [mm]. I made lens disposition measurements in my Micasense RedEdge camera and it happens that these measurements match the pix4d parameters. I post an image to explain that better.

[markwbrown]

@augustinbus yes, I found the same was true with my dual camera setup. Even with rectilineaar lens on action cam, the edges turned out to have a bit of distortion. I cropped down to about 60% of the image (they were shot with 80% overlap) set the red channel to master in Photoscan. Much better results. I still haven’t figured out how to get exif composite gps data on to a two channel tiff without corrupting the tiff. Works fine on one channel and rgb tiff.

[daniopitz]

Hi. Any news about processing multi-spectral images with OpenDroneMap? I have Micasense RedEdge images so I am interested in the best strategy to align the images to create a NDVI orthomosaic.

[smathermather]

This is the closest we have to a solution, at this stage: https://github-production-repository-file-5c1aeb.s3.amazonaws.com/24470090/1944440?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180504%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180504T160336Z&X-Amz-Expires=300&X-Amz-Signature=34dbfa88505fbdb05ceb5cf3e5aec1399c3f3e6821584de2c3ad7361e98e70e0&X-Amz-SignedHeaders=host&actor_id=1174901&response-content-disposition=attachment%3Bfilename%3DProcessingRGBandNIR_InODM_.pdf&response-content-type=application%2Fpdf via @LucasMM86 on this issue https://github.com/OpenDroneMap/OpenDroneMap/issues/500

[smathermather]

It'd be great to have someone script up this process!

[agustinbus]

I can't access the file

[poynting]

For RedEdge data, MicaSense offers some code to get users started on radiometric processing. It's possible this could be adapted for use in ODM. There are examples for automatically detecting reflectance panel images and using that calibration to convert flight images to reflectance, as well as an example showing how to distortion correct and align images and export aligned images to a multi-layer tiff while removing the unmatched overlap.

https://github.com/micasense/imageprocessing

Full disclosure: I'm one of the authors and I work at MicaSense.

[MattWenham]

Hello, i was working with the camera Parrot Sequoia. It has 4 bands. The output of this camera is one tiff image for each band (4 images for each shot), 16 bits resolution. I wanted to do NDVI orthophoto (only 2 bands RED and NIR are needed). I did this steps:

1. make jpg images with RED and NIR channels, with a simple python routine. In this step I had to reduce the resolution 16bits -> 8 bits :(
2. run ODM with the jpg images.
3. calculate NDVI with the ortophoto, with a python routine.

The results are OK as you can see, but i guess it is better to calculate NDVI with the 16bits channels.

One way to do NDVI with better data might be to convert RED and NIR to 12bits each and then spread across the 24bits of an RGB PNG file. In order to maintain image features across all three PNG channels, some kind of ordered packing regime would be needed such as:

12-Bit Data:

Channel {B  A  9  8  7  6  5  4  3  2  1  0} <-- bits
  Red   RB RA R9 R8 R7 R6 R5 R4 R3 R2 R1 R0
  NIR   NB NA N9 N8 N7 N6 N5 N4 N3 N2 N1 N0

8-Bit Data:

Channel {7  6  5  4  3  2  1  0} <-- bits
  Red   NA R9 N7 R6 N4 R3 N1 R0
 Green  RB N9 R8 N6 R5 N3 R2 N0
  Blue  NB RA N8 R7 N5 R4 N2 R1

[tgmueller]

Is there a reliable resolution yet for processing parrot sequoia dat with ODM?

[smathermather]

There is ongoing work due Q1 2020: https://community.opendronemap.org/t/tiff-multispectral-support/3232

[pierotofy]

This is now available in master! Woohoo! :+1: