Add support for multi-layer 16bit TIFFs

[YungKC]

How did you install OpenDroneMap? (Docker, natively, ...)?

Docker

What's your browser and operating system? (Copy/paste the output of https://www.whatismybrowser.com/)

Chrome 88 on Windows 10

What is the problem?

OpenMVS causes ODM to crash while trying to process TIFF files

What should be the expected behavior? If this is a feature request, please describe in detail the changes you think should be made to the code, citing files and lines where changes should be made, if possible.

Should not crash. The original JPG file processed as expected.

How can we reproduce this? (What steps did you do to trigger the problem? What parameters are you using for processing? If possible please include a copy of your dataset uploaded on Google Drive or Dropbox. Be detailed)

This was referenced by another user before: https://community.opendronemap.org/t/error-processing-tiff-images/6276

I experienced the same error message at the start of the dense reconstruction phase via OpenMVS.

[YungKC]

I confirmed that this bug occurs only for 16 bits TIFF files. 8 bit version of the same images worked fine.

Below is my relevant logs:

[INFO] Running dense reconstruction. This might take a while. [INFO] running /code/SuperBuild/install/bin/OpenMVS/DensifyPointCloud "/var/www/data/aad740a6-0c3a-4773-aeea-8e7fc3a79c30/opensfm/undistorted/openmvs/scene.mvs" --resolution-level 2 --min-resolution 320 --max-resolution 1357 --max-threads 12 --number-views-fuse 2 -w "/var/www/data/aad740a6-0c3a-4773-aeea-8e7fc3a79c30/opensfm/undistorted/openmvs/depthmaps" -v 0 07:56:46 [App ] Build date: Feb 10 2021, 20:53:56 07:56:46 [App ] CPU: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz (12 cores) 07:56:46 [App ] RAM: 12.34GB Physical Memory 4.00GB Virtual Memory 07:56:46 [App ] OS: Linux 5.4.72-microsoft-standard-WSL2 (x86_64) 07:56:46 [App ] SSE & AVX compatible CPU & OS detected 07:56:46 [App ] Command line: /var/www/data/aad740a6-0c3a-4773-aeea-8e7fc3a79c30/opensfm/undistorted/openmvs/scene.mvs --resolution-level 2 --min-resolution 320 --max-resolution 1357 --max-threads 12 --number-views-fuse 2 -w /var/www/data/aad740a6-0c3a-4773-aeea-8e7fc3a79c30/opensfm/undistorted/openmvs/depthmaps -v 0 07:56:46 [IO ] error: unsupported TIFF image 07:56:46 [App ] error: failed loading image header 07:56:46 [App ] error: failed reloading image '/var/www/data/aad740a6-0c3a-4773-aeea-8e7fc3a79c30/opensfm/undistorted/images/DJI_0096-2.tif.tif' 07:56:46 [App ] error: invalid project Traceback (most recent call last): File "/code/run.py", line 69, in app.execute() File "/code/stages/odm_app.py", line 83, in execute self.first_stage.run() File "/code/opendm/types.py", line 360, in run self.next_stage.run(outputs) File "/code/opendm/types.py", line 360, in run self.next_stage.run(outputs) File "/code/opendm/types.py", line 360, in run self.next_stage.run(outputs) [Previous line repeated 1 more time] File "/code/opendm/types.py", line 341, in run self.process(self.args, outputs) File "/code/stages/openmvs.py", line 60, in process system.run('%s "%s" %s' % (context.omvs_densify_path, File "/code/opendm/system.py", line 79, in run raise Exception("Child returned {}".format(retcode)) Exception: Child returned 1

[pierotofy]

Hey @YungKC :hand: can you upload these images somewhere? Technically OpenMVS should support 16bit TIFFs (I've added that feature myself). https://github.com/cdcseacave/openMVS/pull/625 but perhaps there's some other issue at play here. Either way, the images would help a bunch.

[YungKC]

Here are the first few files. I also included the first file in 8 bit TIFF format that processed fine for me.

https://drive.google.com/drive/folders/1tsSYEG1Lm6MWy3N_esarkzFg0i7IdS9a?usp=sharing

Kai

On Sat, Feb 27, 2021 at 12:45 PM Piero Toffanin notifications@github.com wrote:

Hey @YungKC https://github.com/YungKC ✋ can you upload these images somewhere? Technically OpenMVS should support 16bit TIFFs (I've added that feature myself). cdcseacave/openMVS#625 https://github.com/cdcseacave/openMVS/pull/625

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OpenDroneMap/ODM/issues/1235#issuecomment-787132611, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADWZ5AD5FDCPZVVSL4K3X3TBFKWJANCNFSM4YJVTQXA .

[zgyivi]

Hi, I have seen similar problem: posted on community page I tried it with a sample image set, which was shared there earlier. In my case it failed on an originally jpeg file: failed reloading image '/var/www/data/9357efe3-2a90-4fc2-b82c-be94813d33fb/opensfm/undistorted/images/DJI_0280.JPG.tif'

[pierotofy]

I see; the problem is that these images are multi-layer 16bit TIFFs. ODM currently has support for single-layer 16bit TIFFs only.

A workaround is to either convert them to 8bit JPG, or separate the bands into individual files (e.g. 1 file per layer, so 3 files per photo).

A substantial effort is needed to add support for multi-layer 16bit TIFFs (needs upgrades to OpenMVS, mvs-texturing and odm_orthophoto, possibly others). We'd welcome contributions to help push this effort faster (either code or financial contributions).

[zgyivi]

In my case I think this is not the root cause, however this improvement would solve my issue too. My original .jpg files were 8bit files, but were converted to 16bit TIFF during undistortion. I think I should create another issue because this is a different problem.

[pierotofy]

I'm not sure what's going on then; could you please:

1) Attach the actual original images 2) The parameters you use to process them in ODM

?

[zgyivi]

Added as new issue: Support of 8bit multi band JPEG + 16bit single band TIFF images #1238

[and-viceversa]

@pierotofy For this issue is it possible to create only the orthomosaic from multiband u16 tiffs in ODM? I'm hoping that might be easier than the textured mesh.

PIL and OpenCV can't open them.

My thought was to edit get_image_size.py to add rasterio for a quick fix:

from PIL import Image
import cv2
import rasterio
import sys

from opendm import log

Image.MAX_IMAGE_PIXELS = None

def get_image_size(file_path, fallback_on_error=True):
    """
    Return (width, height) for a given img file
    """

    # PIL
    try:
        with Image.open(file_path) as img:
            return img.size[0], img.size[1]
    except Exception as e:
        if fallback_on_error:
            log.ODM_WARNING("Cannot read %s with PIL, falling back to cv2." % file_path)
            log.ODM_WARNING("PIL Warning: %s" % str(e))
            pass

    # OpenCV
    try:
        with cv2.imread(file_path) as img:
            return img.shape[1], img.shape[0]
    except Exception as e:
        if fallback_on_error:
            log.ODM_WARNING("Cannot read %s with PIL or OpenCV, falling back to rasterio: %s" % (file_path, str(e)))
            log.ODM_WARNING("OpenCV Warning: %s" % out.capturedtext)
            pass

    # rasterio
    try:
        with rasterio.open(file_path) as img:
            return img.width, img.height
    except Exception as e:
        log.ODM_WARNING("Cannot read %s with PIL, cv2, or rasterio. %s" % (file_path, str(e)))
        raise e

However testing on a u16 6 band stack with --skip-3dmodel --fast-orthophoto to skip texturing still yielded only a RGBA ortho. Having a hard time finding where the extra bands get dropped. I'm curious why this is... it seems like the extra bands are simply "along for the ride" and wouldn't influence much if you're just creating an ortho.

[Matesanz]

I would also add rasterio as fallback when cv2 and PIL fail. I like the solution offered by @and-viceversa. As she/he says it will yield RGBA picture but at least ODM won't crash.

[pierotofy]

Couldn't reproduce with a specific dataset, but might be fixed with #1512 :clinking_glasses:

Please re-open a new issue with a small test dataset if the problem persists. :pray: