DDMAL / pixel_wrapper

Rodan Pixel.js Wrapper
Other
0 stars 1 forks source link

PNG (RGB) job is not handling correctly the output of pixel #86

Closed martha-thomae closed 1 year ago

martha-thomae commented 1 year ago

For some unexplainable reason, when re-uploading the unzip output of pixel and passing it through the PNG (RGB) job, the transparent pixels are transforme into black pixels (rather than white pixels). This messes up other processes like the connected component analysis when using the music symbol layer because all the background transparent pixels are converted by the PNG (RGB) job into black pixels (just like the note symbols) and everything in the layer seems connected and, therefore, cannot be segmented into individual glyphs to be processed by the interactive classifier.

martha-thomae commented 1 year ago

Maybe adding the extra ports for the output layers (see issue #85) might be able to fix this, as the output ports for the layers of the Paco Classifier work fine with the PNG (RGB) job, where the transparent pixels are converted into white.

martha-thomae commented 1 year ago

I have updates regarding this issue now that in staging we can get the layers out of pixel (we have output port for the individual layers).

The issue is:

  1. When the PNG (RGB) job receives a layer that comes out of Pixel, the transparent pixels are converted to black pixels
  2. When the PNG (RGB) job receives a layer that comes out of the Fast Pixelwise Analysis of Music Document, Classifier, the transparent pixels are converted to white pixels

The second one (conversion of transparent pixels into white pixels) is the behaviour we want. I don't know why the behaviour is different when the layers come from pixel. I am including an example here.

1) Pixel workflow:

screenshot with pixel

A) Output of pixel for the staff line layer:

1_Pixel_js - PNG - Layer 1 Output-2

B) Then the result of passing it through PNG(RGB) job:

2_PNG (RGB) - RGB PNG Image-9

C) And then the result of passing this into the convert to black and white job (it looks like the negative):

3_Convert to one-bit (black and white) PNG - One-bit PNG image-8

If you want to try it yourselves, here is the workflow to use in staging (you can uncompress the file, and then import the workflow into a project rodan):

ONLY_PNG(RGB) with Output Layer from PIXEL.json.zip

2) With the Fast Document Analysis workflow:

screenshot with fastdocanalys

A) Output of the Fast Pixelwise Analysis of Music Document, Classifying for the staff line layer:

1_Fast Pixelwise Analysis of Music Document, Classifying - Layer 2

Then the result of passing it through PNG(RGB) job:

2_PNG (RGB) - RGB PNG Image-8

B) And then the result of passing this into the convert to black and white job (it looks good!):

3_Convert to one-bit (black and white) PNG - One-bit PNG image-7

C) If you want to try it yourselves, here is the workflow to use in staging (you can uncompress the file, and then import the workflow into a project rodan):

ONLY_PNG(RGB) with Output Layer from FASTPIXEWISEANALYSISOFMUSICDOC Model.json.zip


As you can see, the input of both workflows looks the same (items A), but the difference occurs already in the PNG (RGB) job where the transparent pixels are converted to black (item 1B) or to white (item 2B). The expected behaviour is as item 2B, but I don't know why the png(rgb) job is working differently for these two images (items A), which look exactly the same.

This issue is high priority, as the wrong behaviour of pixel makes the future job of CC Analysis to obtain wrong glyphs (as it detects the whole page as a huge connected component, given its treatment of black pixels). Could you look into it @timothydereuse @sabrina0822 @softcat477 ? The issue might be with pixel, or it might be with the PNG(RGB) job.

malajvan commented 1 year ago

I was able to recreate this on develop branch locally, using only the provided jobs' output Martha provided above and straight into PNG (RGB).
Both files are rgba-png formats being converted in rgb-png by PNG (RGB) job, which is a simple wrapper for the PIL function "convert('RGB')" image by converting from rgba to rgb, we're discarding the alpha channel (a in rgba) therefore there will be problems handling them. Not sure how (or if) the jobs worked before. Working on a fix.

malajvan commented 1 year ago

@martha-thomae I've made the fix in the PR above, we'll let you know when it's merged and on staging/prod!

martha-thomae commented 1 year ago

Thank you so much, @malajvan! Yes, please let me know when this is on staging for me to test it, so that we can close this issue and then put the change in production for the Alamire research group to use it. Thanks!

martha-thomae commented 1 year ago

I tried the workflow again in rodan.staging.simssa.ca, and the issue is fixed! Thank you, @malajvan!

martha-thomae commented 1 year ago

I think this is ready to be moved into production (both this fix in the PNG(RGB) job and the one that allows for optional output port layers in pixel). These fixes will be useful for the people from the Alamire project. @timothydereuse