search

martinbenson / deep-photo-styletransfer

Implementation of "Deep Photo Style Transfer": https://arxiv.org/abs/1703.07511
MIT License
204 stars 27 forks source link

Segmentation/masking temporarily removed #26

Closed martinbenson closed 7 years ago

ProGamerGov commented 7 years ago

How far have you gotten so far, on re-adding segmentation/masking?

Do you think multi style image support possible with minimal alternations? The other issue, is that it is that the color selection code could be done in a way that makes it easier to add new colors, as per luanfujun here.

During my attempts to get segmentation working in the newer Neural-Style format, I noticed that some parts of the segmentation code were not used, and seemed to be from the creator's experiments that they did not clean up.

ProGamerGov commented 7 years ago

For anyone needing to use segmentation masks, you can use: git checkout 262a825 to change the version of your directory back to the old version with the working masks.

martinbenson commented 7 years ago

I decided to do it in two parts - adopt Johnson's new code without sorting out the masking part, and then after sort that out. It didn't look too hard, so am hopeful that i can do it next time i have a few hours - unless someone beats me to it!

ProGamerGov commented 7 years ago

@martinbenson Going beyond the scope Deep Photo Style Transfer research paper, another somewhat recently released research paper addresses some issue with segmentation in regards to style transfer. Because these two papers seem to have been published within a short period of time of each other, the work done in either could not have been used in the other's code.

I don't know how much of an issue the receptive field problem in regards to Spatial Control (style transfer with segmentation) is with deep-photo-styletransfer, but Leon Gatys' NeuralImageSynthesis (the code from his research paper) solves the issue via adding in place dilation to the style loss function. Specifically, the relevant code is in the LossLayers.lua file. In the research paper, I believe that the in-place dilation is called "eroded guidance channels", where "guidance channels" refers to inputs and settings that affect your output.

DenisSergeevitch commented 7 years ago

Wow, thats why I cannot use "-content_seg" option

martinbenson commented 7 years ago

I have now re-implemented segmentation/masks!