Saturation Artifacts Python

[A-Jacobson]

What was the issue with the torch image package that caused the Saturation Artifacts?

I'm implementing fast neural style and super resolution in pytorch with pillow 4.3 and python 3.6 and am getting similar artifacts. I have tried pillow-simd and several versions of normal pillow and the problem persists. Hoping this may give me a hint.

[ProGamerGov]

Are you by any chance using some form of histogram matching, or grey-scale related stuff?

Because it sort of looks like the issue I experienced here: https://github.com/jcjohnson/neural-style/issues/376#issuecomment-282117186

[A-Jacobson]

Nope, I just load the images and divide by 255. Then before the outputs go through vgg in the loss function I’m subtracting means and dividing by std as specified by the pre trained network.

I didn’t see anything similar to this in the thread you linked. What were you referring to? On Fri, Oct 6, 2017 at 2:44 PM ProGamerGov notifications@github.com wrote:

Are you by any chance using some form of histogram matching, or grey-scale related stuff?

Because it sort of looks like the issue I experienced here: #376 (comment) https://github.com/jcjohnson/neural-style/issues/376#issuecomment-282117186

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jcjohnson/neural-style/issues/427#issuecomment-334877577, or mute the thread https://github.com/notifications/unsubscribe-auth/AI-bzFjAKH9nyTISlFH9LzLEtuVp4-34ks5spp89gaJpZM4PwK3s .

[ProGamerGov]

@A-Jacobson Without seeing your code, or the original input images, it's hard to tell what's causing the artifacts.

What was the issue with the torch image package that caused the Saturation Artifacts?

Was this related to an issue posted on this GitHub project, another project, or something else? And if so should it be searchable?

As for linking to the thread, I thought the artifacts looked like those caused by extremely bright or dark pixels in the input images, that occurred while trying to replicate Gatys' research paper's code, in Neural-Style. However I don't think your images look like they have any issues with extremely dark or bright colors.

[A-Jacobson]

Was this related to an issue posted on this GitHub project, another project, or something else? And if so should it be searchable?

Yes if you check the readme and issue https://github.com/jcjohnson/neural-style/issues/18. Looks like a similar problem. Though I'm using pytorch and PIL not the LUA image package.

Style Transfer code: https://github.com/A-Jacobson/paper_implementations/blob/master/Fast_Style_Transfer/Style%20Transfer.ipynb

Super resolution code: https://github.com/A-Jacobson/paper_implementations/blob/master/Super_Resolution/4x%20Super%20Resolution.ipynb

[htoyryla]

As far as I know, that issue was with an older version of lua image package, so I don't think it is relevant now.

The excellent pytorch tutorial on neural-style at http://pytorch.org/tutorials/advanced/neural_style_tutorial.html says:

"However, there’s a small catch. The optimized image may take its values between −∞ and +∞ instead of staying between 0 and 1. In other words, the image might be well optimized and have absurd values. In fact, we must perform an optimization under constraints in order to keep having right vaues into our input image. There is a simple solution: at each step, to correct the image to maintain its values into the 0-1 interval."

They implement this by clamping the input between 0 and 1 in the optimization iteration.

BTW, this pytorch/neural-style tutorial is also a very clear and detailed, yet compact explanation of the neural-style process.

PS. Your python notebook appears to be about fast-neural-style which is a different process (training a network instead of optimizing an image). However, the point of keeping the correct value range during optimization might still be a valid point.

[A-Jacobson]

My network ends in a tanh so it should be bounded between -1 and 1. I'm aware or the difference between fast neural style and normal neural style. I was just hoping to get a hint from this issue as it is the only thing I have seen that is similar to the bug I am encountering and was apparently common enough to make it into the readme.

[htoyryla]

Maybe it is the negative values that's causing the artifacts?

The issue with image package got into the readme as the old version was still common at that time, I guess. No-one remembers any longer what the issue in image package was. You can of course go and browse through its change history but my bet is that it's got nothing to do with your problem.

[A-Jacobson]

could be! I'm currently preoccupied with another model but will certainly add that to my list of painfully obvious things to check once i get back on that project. thanks for the ideas.

[A-Jacobson]

Turns it out was the negative values. A simple .clamp(0,1) or sigmoid rather than tanh fixed the problem. Thanks for the help!