Implement user-selection classification regions for patchwise classifier

[EricHanLiu]

For the patchwise method we need the user to predetermine which region they will classify in the image, and have them thoroughly classify every pixel within that region into one of the 3 layers (symbols, staff lines, text).

The user can have multiple such selections in a score. The background should only be generated within these regions, and the rest of the image will not have any layer data.

• [x] Implement the new select region tool which generates a rectangle in those regions which the user is confined to (or have the default layer be a new colour which acts as the select region layer by using the rectangle rool, ensure user does not do anything else with this layer)
  • [x] Fix bugs with selectRegionLayer and creating/deleting layers (which is only possible if there are no input ports)
  • [x] add instructions on how to use this layer in the tutorial
• [x] Have the background layer only be generated within those regions
  • [x] ignore rectangles that were erased

[EricHanLiu]

Set the default layer (layer 1) to be the select tool, maybe disable all other tools except the rectangle tool if layer 1 is active. Have the color be close to transparent with maybe a border (if possible to implement).

Generate the background layer based off this rectangle, so maybe something like

background.addShapeToLayer(layer1.shapes)

and then the background will just be these shapes, and only iterate over the image pixels that are contained in these rectangles

[EricHanLiu]

Background layer is now always generated within the select region(s). Right now it iterates over the whole image, so it still takes as long as before. I'd like it to only iterate over the select regions, however this becomes difficult if there are multiple such regions and they have some overlap. I'd have to do a forEach loop over the rectangles (select regions), but I'd need to change the way the progress bar is generated.

Not a pressing issue though since the user will have to wait for the CM anyway, so what's an extra 2 minutes of waiting...

Will work on deactivating the other tools if in the selectRegionLayer

See branch

[EricHanLiu]

Tried deactivating the other tools and I managed to remove the non-rectangle-tool buttons when the selectRegionLayer is highlighted, and they get readded when selecting a different layer. However, since the user can use hotkeys to get the different tools, just removing the buttons isn't enough (if the user hotkey-selects the brush tool while in selectRegionLayer, pixel crashes).

In order to implement this properly I'd have to edit the standalone Pixel.js quite a lot for wrapper-specific features, which isn't ideal. I'll stick to adding to the tutorial and telling the user not to use other tools in that layer (although they wouldn't affect the background generation)

[EricHanLiu]

Most issues with selectRegionLayer are solved by closing Pixel.js after the user Submit To Rodan.

Issues arose when the user would Submit To Rodan, then create or remove layers and edit the image more (despite the fact that they couldn't resubmit).

These issues included:

• the backgroundLayer being editable after creating/deleting a new layer after submitting
• the backgroundLayer being wrongly generated after submitting at least twice

Though to be honest these are cuased by the user trying to do things they shouldn't (submitting twice)

Hopefully it's okay to force Pixel to close after submitting to Rodan (since the user shouldn't be allowed to re-edit anything, if they do it's for nothing)

[EricHanLiu]

Optimization seems like it can't actually work, since there are many rectangles that are created in the selectRegionLayer that shouldn't be iterated over. For example, any subtract rectangles that were drawn (right click drag) randomly, or portions of actual rectangles that were erased. There's no way to retrieve these exact rectangles without checking for pixel-opacity, which is how the actual background generation works now.

It would still be comparably slow to check the actual rectangle bounds that we need to iterate over, and then only iterate within these rectangles.

[EricHanLiu]

Created a prompt for the user to input the number of layers they'll create when they start Pixel.js, which then get generated. The create/delete layer buttons have been completely removed for the wrapper. Added a bit of info on how to use the selectRegionLayer in the tutorial

I've moved the instantiation of the wrapper to the end of the plugin activation, so that the tutorial could be altered (and this also allows the Submit To Rodan to be positioned much nicer than before)

[EricHanLiu]

So basically this feature is done, we just need to see what direction we'll go into with the CM and how the background should actually be generated.

• [x] Implement selectRegionLayer with rectangle default tool and borders
• [x] Only generate background within this layer
• [x] Disable create/delete layer buttons
• [x] Generate the number of layers by # of input ports OR prompt user for input
• [x] Close Pixel.js after submitting to Rodan. This might not be kept though, it was originally implemented to solve the issues in the emoji-marked comment, but one of those was solved by removing the create/delete layer functionality.
• [x] Add small tutorial update with selectRegionLayer usage instructions

[EricHanLiu]

pull request accepted