Help needed: layers sometimes completely empty

[JoyfulGen]

There are two things that happen:

• A layer is completely empty because all of its elements ended up in another layer; Like so:

  

• A layer is completely empty because all of its elements are still in the background. Like so (you can see that the staff lines and text were extracted, but not the glyphs at all):

  

• (There was also one time where all three layers were completely empty because everything was in the background).

This phenomenon has happened at the beginning of iteration or later down the line, with cropped images or full images, with iteration or without. I'm making this issue because I'm not sure where to go from here and I want to make sure to have a record of it.

These models produce all empty layers (so no layer separation at all): Background Model MTWS 023 round 3.hdf5.zip Model 1 MTWS 023 round 3.hdf5.zip Model 2 MTWS 023 round 3.hdf5.zip Model 3 MTWS 023 round 3.hdf5.zip

These models produce an empty layer 1: Beatrice 6+7 Background Model.hdf5.zip Beatrice 6+7 Model 1.hdf5.zip Beatrice 6+7 Model 2.hdf5.zip Beatrice 6+7 Model 3.hdf5.zip

And these models produce empty layers 1 and 3 (so only layer 2 gets separated): Beatrice 7 part 2 Background Model.hdf5.zip Beatrice 7 part 2 Model 1.hdf5.zip Beatrice 7 part 2 Model 2.hdf5.zip Beatrice 7 part 2 Model 3.hdf5.zip

[JoyfulGen]

Ok! I have a more concrete example. I started by making layer models with this score:

I categorized my layers the usual way:

• Neumes and clefs go in layer 1, blue
• Staff lines go in layer 2, red
• Text goes in layer 3, yellow

Those models produce almost impeccable layers 1 and 2, but they put all of the words in layer 1 as well, which means that layer 3 is completely empty. You can see this on this Pixel screenshot. The clefs and notes are blue and the staff lines are red, which is correct. However, the words should be yellow, but they're blue, and there's no yellow anywhere:

I then iterated those models, to try to improve the results. The new models I ended up with do indeed put text in layer 3, but they also do much worse on both staff lines and notes. You can see that there's a lot of blue on the staff lines, and quite a bit on the words as well, and there's a lot of yellow in the notes:

@fujinaga, is this expected behaviour?

[kyrieb-ekat]

I was able to replicate this issue, where the third layer, which ought to have been entirely text, appeared empty. However on closer examination it appears that only a pixel or two makes it through. See attached images (layers 3 are the problem layers)- there are a couple small specks present so they're not wholly empty. The groupings of the images below are the layers 0 (background), layers 1, layers 2, and layers 3.

Interestingly the very edge of the image being included has been an observed behavior before, where the edges eventually never get pulled through for any interactive layer (i.e., layers 1-3 and always remain in L0/background).

[JoyfulGen]

Amazing! @fujinaga we have results that can be reproduced. The following layer separation models produce an (almost) empty layer 3 every time:

Paco Test 13 Background Model.hdf5.zip Paco Test 13 Model 1.hdf5.zip Paco Test 13 Model 2.hdf5.zip Paco Test 13 Model 3.hdf5.zip

This is the workflow: Fast.json

And here are a couple of images to use the models on: