Deployed Version of Panel App Not Working with Orthomosaics

[2320sharon]

Deployed Version of Panel App Not Working with Orthomosaics

I was trying to get the deployable version of the application to work with some orthomosaics after verifying it worked in jupyter lab. But the application won't let me doodle, change the image displayed or click any of the buttons, including the advanced checkbox. Everything looks okay, but I currently have 0 functionality for the deployed version.

I can verify that the deployable version works with the default images and classes, but not with orthomosaics currently.

Terminal Output

(doodler-dev) C:\USGS\CoastSeg\repos\HoloDoodler_fork\repo\holodoodler>panel serve doodler.ipynb --show 2022-01-10 13:05:13,289 Starting Bokeh server version 2.3.3 (running on Tornado 6.1) 2022-01-10 13:05:13,304 User authentication hooks NOT provided (default user enabled) 2022-01-10 13:05:13,308 Bokeh app running at: http://localhost:5006/doodler 2022-01-10 13:05:13,308 Starting Bokeh server with process id: 13756 2022-01-10 13:05:29,608 W-1002 (EMPTY_LAYOUT): Layout has no children: Column(id='1218', ...) 2022-01-10 13:05:30,375 WebSocket connection opened 2022-01-10 13:05:30,378 ServerConnection created 2022-01-10 13:05:34,661 WebSocket connection closed: code=None, reason=None

The Offending Orthomosaic

[jbednar]

I'd suspect it's not finding the image at the location where it's looking; check the paths?

[2320sharon]

I tested it with other images from other datasets like NOAA and Landsat and it works normally. But when I test it with any orthomosaics it fails. I tried testing it with a different orthomosaic and I got a different behavior. The buttons worked but all the colors were inverted. I must say I'm confused by this behavior as well.

[EDIT] I tested the same orthomoasics in jupyter lab and they can be segmented, but they also have an inverted color scheme.

[jbednar]

Jupyter and Bokeh Server have different policies for what files they can read; I believe Jupyter can only read subdirectories of where the app was launched, while Bokeh (and thus Panel) Server can read them relative to where the app is located. This is usually the cause of differences in file-finding behavior between the two cases, but here that would only explain it if your two orthomosaic types were in different directories. Not sure what's up with the colors, and in any case if the different behavior in the server is to do with paths we need a better error message.

[dbuscombe-usgs]

If I remember correctly, this is something I have encountered before when working with RGB images with missing data and alpha layers

This is why dash-doodler uses imread from scikit-image (from skimage.io import imread in this) instead of PIL.Image to read images

then in this, images are read and the 4th band (i.e. alpha) is removed

            orig_image = imread(select_image_value[0])
            if np.ndim(orig_image)>3:
               orig_image = orig_image[:,:,:3]

Hope thats helpful. I understand if you dont want an extra dependency (skimage), but it does have a lot of image backends that could be used

[jbednar]

Thanks! skimage is a fairly heavy dependency we try to avoid in general, but I think it's reasonable to use it here, either as a required dependency or by first checking if skimage is installed.

[maximlt]

I downloaded the last JPEG you shared @2320sharon and ran image magick to get more info:

❯ magick identify examples/images/orthomosaic2.jpg
examples/images/orthomosaic2.jpg JPEG 1024x1024 1024x1024+0+0 8-bit CMYK 273549B 0.000u 0:00.001

So this sort of JPEG has 4 bands based on the CMYK model, and not RGB + alpha. This is why removing the last band as I did messed up the color of these images 🙃

Using Pillow's convert() method seems to do the trick. As you can see the image in the middle (the one obtained by converting it with Pillow) looks exactly the same as the one on the right (that is just loaded in the browser), the one on the left is obtained by stripping the last band (which is what we currently do and is obviously wrong):

I think this is the change I will implement since I was using Pillow already and it seems straightforward. But as you suggest @dbuscombe-usgs it seems it could be handled by skimage, which is anyway a dependency of Doodler's inner algorithms.

[dbuscombe-usgs]

Nice solution! I never thought to use Pillow's convert method, thanks for the tip!