search

DDMAL / pixel_wrapper

Rodan Pixel.js Wrapper
Other
0 stars 1 forks source link

Pixel/CM Summer 2018 overview #5

Open vigliensoni opened 6 years ago

vigliensoni commented 6 years ago

Test patch-wise classification method and Pixel

In order to do implement bullet point 5, we need to work on:

EricHanLiu commented 6 years ago

Pixel works as a job in Rodan taking jpg/png/tiff resource and outputs 3 layer resources

vigliensoni commented 6 years ago

@EricHanLiu Did you actually perform a full test? Were you actually able to export all image layers and open them in RODAN?

EricHanLiu commented 6 years ago

All image layers can be exported (either as highlights, image data) and downloaded locally, or sent directly to Rodan to be piped into future jobs (saved as resources). The outputs are rgba+png files.

EricHanLiu commented 6 years ago

On step two we encountered a bug with Keras, and so the new workflow, which contains our patch-wise CM job, cannot finish. The issue is created and I'd like feedback from Jorge before trying a couple workarounds were presented on this issue.

EricHanLiu commented 6 years ago

Jorge suggested it's a version inconsistency error (which I saw mentioned a lot also on the issue page), so I will try to verify which backend Keras is using and play around with the settings to see if that fixes the issue, otherwise maybe deleting the optimizer

EricHanLiu commented 6 years ago

Keras uses Theano like suspected, tried second method which was to remove optimizer weights in each model file using a python script. This should fix the issue and allow us to checkoff the second part (test patch-wise CM with single images in Rodan)

EricHanLiu commented 6 years ago

Patch-wise classification works with single images in Rodan (using the demo pre-trained models)

fujinaga commented 6 years ago

👍

EricHanLiu commented 6 years ago

Using patch-wise CM as a Rodan job and piping the output layers (excluding background) into Pixel in a single workflow works, and the output of patch-wise CM can be corrected in Pixel and resubmitted to Rodan (or exported to CSV/highlight png/image-data png)

EricHanLiu commented 6 years ago

Simultaneously testing adding another resource in the queue for the patch-wise classifier to run after the first pixel image was corrected, will update once it finishes (or segfaults)

martha-thomae commented 6 years ago

Patch-wise CM works with multiple images on the queue, we tried with 3 sections of full TIFF images, it generated the 4 layers (background, staff-lines, music symbols, and text) for each of them correctly.

EricHanLiu commented 6 years ago

Also works when multiple images are input to patch-wise CM, and then piped into Pixel, multiple pixel jobs can be opened to correct classification and then resubmit to rodan

vigliensoni commented 6 years ago

@EricHanLiu One Pixel job at a time in series or many of them in parallel?

EricHanLiu commented 6 years ago

It depends on when the CM job finishes, but Pixel can be opened as soon as the CM job finishes and the last test we did (with two images) they finished simultaneously so two instances of Pixel could be opened with different classifications (although with the same image since Pixel only takes 1 image input in one job, so one of the classifications was off) Since the fast CM doesn't output the original image, we can't load Pixel with the correct image for every job

EricHanLiu commented 6 years ago

Pixel now can export a background layer along with the other 3 layers, with some caveats (see this issue). We've added another port, so the output of Pixel can be piped directly into the fast CM trainer as ground truth.

EricHanLiu commented 6 years ago

Pixel now almost entirely works, only bug that remains is the erasing other layers one. However, multiple exports with edits in between can now be done without error, and export to image data successfully works.

vigliensoni commented 6 years ago

☀️ ☀️ ☀️

EricHanLiu commented 6 years ago

Pixel's background layer is now generated through pixel subtraction instead of path/shape subtraction, so it is now compatible with the Calvo classifier's output. Background layer generation is assigned to a button, so user has the option to generate it.

EricHanLiu commented 6 years ago

Iterative, incremental Rodan workflow depends on some sort of Rodan loop being implemented, as well as the patch-wise classifier being finished.

EricHanLiu commented 6 years ago

selectRegionLayer is implemented with a tutorial tooltip and restrictions on bad usage. Pixel is a submodule so pixel_wrapper is the latest version, but create/delete layer buttons were removed since they shouldn't be features within a workflow.

Number of default layers is generated based on either the number of input ports, or a prompt will ask the user how many layers. The input and output ports for layers are now optional, with max being 7 layers.

The background is only generated within the selection regions, and it has been optimized to only iterate within those regions. Generation has been moved to submitToRodan button and Pixel closes automatically after exporting.

Not sure what else needs to be done for this issue aside from fixing the Docker swap memory bug, and figuring out what exactly we're doing with the CM training

EricHanLiu commented 6 years ago

How should we proceed here with the next checkbox?

Design iterative, incremental Rodan workflow to correct portions of images and reclassify

vigliensoni commented 6 years ago

We've implemented part of the workflow, but we still need to work on:

On Thu, Jun 28, 2018 at 1:06 PM, Eric Han Liu notifications@github.com wrote:

How should we proceed here with the next checkbox?

Design iterative, incremental Rodan workflow to correct portions of images and reclassify

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/DDMAL/pixel_wrapper/issues/5#issuecomment-401106282, or mute the thread https://github.com/notifications/unsubscribe-auth/AAeCxeaYaO_PkqheOM1jf4KDt3dM1h7cks5uBQ0xgaJpZM4Tjlap .