Match the Format of Outputted Images with the Format of Input Image (especifically GeoTIFFs)

[venuswku]

New Feature Request

• Currently, the saved output images for any input image are colorized_segmentation.png, input.jpeg, and doodles.jpeg. @dbuscombe-usgs suggested that it be useful for the format of the outputted images (colorized_segmentation and input) to match the format of the input image.
• Especially when the input is a GeoTIFF, which is a TIFF file embedded with geographic information related to the image, it would be useful if the colorized_segmentation and input images had the same geographic information as the input. When these output images are displayed on a map with GIS software, they will be placed at the map coordinates that the input image is georeferenced to.
  • The doodles image might be used to test other segmentation models, but it's not super useful to view it on a map so I don't think it needs to contain any geographic information. What do you think?
• It would be nice to add an overlay (colorized segmentation on top of the input image) as one of the output files. The format of the overlay image should match the format of the input image since it contains both the colorized_segmentation and input images.

Suggested Solution

• View my modifications for saving output files on the forked branch https://github.com/venuswku/holodoodler/tree/segment-tiff-files. A pull request is linked to this issue.
• I mainly used OSGeo's gdal module and Pillow's Image module to save the colorized_segmentation , input, and overlay images in specific formats.
  • HoloDoodler never used gdal before, so I added it as a new dependency in environment_dev.yaml.
• I doodled randomly for testing so please excuse my inaccurate segmentations below!
• For GeoTIFF inputs, I viewed the output files with leafmap. Can someone check if the output images can be viewed with other GIS software like QGIS? I'll be testing with QGIS once I make some space to download it on my laptop.
  • Example output images displayed on leafmap:
    • input.tif
    • colorized_segmentation.tif
    • overlay.tif
      • It's easier to view overlays for brighter images. This input image is dark, so you need to zoom in to view the segmentation better.
    • Actual GeoTIFF input: samplegeotiff.zip
    • Actual output files for GeoTIFF: 20221020-152250.zip

Notes

• JPG/JPEG inputs: I saved the colorized_segmentation and overlay images as PNGs because JPG/JPEG images don't support alpha/transparency channels. The input image is saved as a JPG/JPEG though!
  • Actual JPG/JPEG input: example1.jpg
  • Actual output files for JPG/JPEG: 20221020-152811.zip
• TIFF inputs (not GeoTIFF): These normal TIFF files don't have geographic information embedded in them, but I still saved colorized_segmentation , input, and overlay as TIFF files. You just can't view them on a map with GIS software.
• The following block of code was used to display the GeoTIFF image above. It scales the pixel values so that the image could be displayed on the HoloDoodler app. I temporarily added this in order to see the image that I'm doodling on, but I will remove this once I find a better way to make different input images compatible with HoloDoodler and its segmentation model from doodler-engine. See more details here: https://github.com/Doodleverse/holodoodler/issues/6.
  • From ./doodler/components.py's _load_image() function:

    # Make sure image array is within the range [0, 255] for integers or [0, 1] for floats.
    if np.issubdtype(img.dtype, np.integer) and not (np.all(img >= 0) and np.all(img <= 255)):
    img = (img / np.amax(img) * 255).astype(np.uint8)
    elif np.issubdtype(img.dtype, np.floating) and not (np.all(img >= 0) and np.all(img <= 1)):
    img = img / np.amax(img)
    # Infinity can only be represented as a float as of right now, so we don't need the following two lines for scaling integers.
    img[img == float("-inf")] = float(0)
    img[img == float("inf")] = float(1)

[dbuscombe-usgs]

Excellent, thanks @venuswku. I like all these changes, and will accept and merge your PR today

I tested the output geotiff files in QGIS. They appear in the correct place, and look good

I like the output json file, "info.json", containing metadata.

There are three further changes I would like to see

The first relates to your question The doodles image might be used to test other segmentation models, but it's not super useful to view it on a map so I don't think it needs to contain any geographic information. What do you think? I think there are good reasons to make the doodles image a geotiff, if the inputs are geotiffs. It would be valuable to see where in space those doodles were made.

The second request is to make only the following outputs

1. colorized segmentation
2. greyscale segmentation - this is the 1-band 8-bit integer or greyscale version of the doodler output, before it gets colorized
3. doodles
4. info json file

The "input" file that is currently an output is surplus to requirements - it merely duplicates the input file

The third request is to name the output files with the same root as the input file. So, if the input file is called samplegeotiff.tif, the outputs should be called 1) samplegeotiff_doodles.tif, 2) samplegeotiff_colorlabel.tif, 3) samplegeotiff_label.tif, and 4) samplegeotiff_info.json. This will be easier to match input files with outputs, based on filename. The info.json files could later be consolidated into a single JSON file containing all metadata info for all doodled images

Thanks!

[venuswku]

Thanks for testing the output files in QGIS, @dbuscombe-usgs!

The info.json file is actually Maxime's idea.😄 I just modified it a little bit to reflect my changes.

I made a pull request for your three changes here: https://github.com/Doodleverse/holodoodler/pull/7. Feel free to check it out when you have time! Here are the new outputs if the input is another_geotiff.tif.zip: 20221021-201048.zip.

• Please excuse the inaccurate segmentation and dark images haha
• another_geotiff_doodles.tif
• another_geotiff_label.tif (grayscale)
• another_geotiff_colorlabel.tif
• another_geotiff_info.json

  {
     "time": "20221021-201048",
     "user": "placeholder",
     "settings": {
         "crf_downsample_factor": 2,
         "crf_mu": 1,
         "crf_theta": 1,
         "edges": true,
         "intensity": true,
         "multichannel": true,
         "n_sigmas": 2,
         "rf_downsample_value": 1,
         "sigma_max": 16,
         "sigma_min": 1,
         "texture": true
     },
     "classes": [
         "no_data",
         "whitewater",
         "water",
         "sand",
         "grassy",
         "woody",
         "green_marsh",
         "gray_marsh",
         "anthro"
     ],
     "colormap": [
         "#D3D3D3",
         "#d8e8ed",
         "#0f5e9c",
         "#deb42a",
         "#C5e946",
         "#997548",
         "#169156",
         "#8fa89c",
         "#af4fa2"
     ],
     "input": {
         "image": "examples\\images\\Klamath river mouth\\another_geotiff.tif"
     },
     "output": {
         "doodles": "results\\20221021-201048\\another_geotiff_doodles.tif",
         "label": "results\\20221021-201048\\another_geotiff_label.tif",
         "colorlabel": "results\\20221021-201048\\another_geotiff_colorlabel.tif"
     }
  }