jcjohnson / neural-style

Torch implementation of neural style algorithm
MIT License
18.3k stars 2.71k forks source link

Has anyone been able to match Dreamscope results? #257

Open ProGamerGov opened 8 years ago

ProGamerGov commented 8 years ago

I can't seem to get close. If you have, what sort of settings are you running? I shamelessly copied the question from https://github.com/jcjohnson/neural-style/issues/240

Runescaped commented 8 years ago

Do they reveal their style images? If so, post an example and I'll try mah best.

ProGamerGov commented 8 years ago

@Runescaped What do you mean? Unlike Ostagram, its free everything, and you do not need a social media account to sign in. You get to choose a custom style image or an existising style image. You supply the content image and the style image.

https://dreamscopeapp.com

The "Jewel" style is just the triangle style image here: https://i.imgur.com/j79FllS.jpg

Runescaped commented 8 years ago

You get to choose a custom style image or an existising style image. You supply the content image and the style image.

I visited the site, but didn't feel like registering an account. I was expecting something similar to ostagram, where they show the content and style images with the result.

I'll try to generate the final image in this album. This example was grabbed off a random user's page.

ProGamerGov commented 8 years ago

@Runescaped These are albums of images I have made on Dreamscope along with the style and content images:

Style Album 2: https://imgur.com/a/3FPbj

Style Album 3: https://imgur.com/a/yBoHz

Style Album 4: https://imgur.com/a/3XBS3

Style Album 5: https://imgur.com/a/pNDyT

Style Album 6: https://imgur.com/a/fOb1f

Space Cat Style: https://imgur.com/a/fMyBG

Justin Trudeau Weed: https://imgur.com/a/VsVsw

Space Flower Style: https://imgur.com/a/WuVQw

Fire Line Style: https://imgur.com/a/0Oafv

Ice Style: https://imgur.com/a/4MQXw

More Ice Style Styles: https://imgur.com/a/QTn9o

Painted Swirl Style: https://imgur.com/a/b0G08

Bright Colored Elephant: https://imgur.com/a/uVk3t

Style Effects on a Drawing: https://imgur.com/a/nLNsx

Ice, Magma, and LHC Style: https://imgur.com/a/M14m3

Interesting Style: https://imgur.com/a/JCChb

Lighting Experiments: https://imgur.com/a/cVT1g

Quilling Paper: https://imgur.com/a/vbd2s


I can provide more if needed.

The content images I use can be found here on their own:

https://imgur.com/a/Gvjjb

And the drawn women content image can be found here: https://i.imgur.com/IDl53R8.jpg

ProGamerGov commented 8 years ago

@Runescaped Also see

Lucid Lynx: https://imgur.com/a/4ZM6G

Pixie Cold: https://imgur.com/a/e9to0

3DTOPO commented 8 years ago

Have you tried CNNMRF?

https://github.com/chuanli11/CNNMRF

The results look like it to me.

ProGamerGov commented 8 years ago

@3DTOPO

Using the content images with the style images on that Github on the Dreamscope site, gives me these results: https://imgur.com/a/QO7Bu

The talk of color independent style transfer and other comments by the creators seems to imply Neural-Style is used.

Dreamscope creator: https://www.reddit.com/user/sabalaba

Other Co-Creator: https://www.reddit.com/user/mippie_moe

3DTOPO commented 8 years ago

They look pretty similar. There are a ton of options. Even more possibilities than neural-style.

Runescaped commented 8 years ago

Looks like you got your answer: https://www.reddit.com/r/deepstyle/comments/4p1mh1/an_old_illustration_of_mine_stylized_in_dreamscope/d4hn0ys

3DTOPO commented 8 years ago

I don't know - sounds like they aren't very sharing of the details. ;)

I mean - could they be any more vague?

Runescaped commented 8 years ago

Sorry for the late response, here's the results I got when dealing with the dog.

Not surprisingly, I wasn't able to get very similar results to the Dreamscope one.

ProGamerGov commented 8 years ago

@Runescaped Though on the bright side some of those images have interesting interpretations of the style.

ProGamerGov commented 8 years ago

@3DTOPO CNNMRF does not seem to give similar results to dreamscope.

3DTOPO commented 8 years ago

It was just a gut feeling. That said - there are practically an infinite number of variations possible.

sdziscool commented 8 years ago

Have you tried using https://github.com/awentzonline/image-analogies instead of CNNMRF and neural style? one of the examples looks really like a dreamscope or ostagram result.

jup

ProGamerGov commented 8 years ago

I created a new wiki page in the hopes that eventually someone will link to something that produces similar results to Dreamscopeapp: https://github.com/jcjohnson/neural-style/wiki/Similar-to-Neural-Style

ProGamerGov commented 8 years ago

Does anyone know any other systems which would create similar effects to Dreamscopeapp?

Or maybe it's a different model they are using? There are some pretty big differences on outputs between models.

ProGamerGov commented 8 years ago

Let's talk about this one brain everyone loves. It's a bvlc_googLeNet trained on ImageNet, provided on the caffe model zoo. That's the one that gives us puppyslug because it has seen so many dogs, birds, and pagodas. It's also the one that gives you the rest of the effects offered by dreamscopeapp because they're just poking the brain in other places besides the very end. Again, even the deluxe options package is going to get old fast. I refer to this caffemodel file as the puppyslug brain. Perhaps the reason for all the doggies has to do with the number of dog pictures in ImageNet. Shortly following is a diagram of the images coming from different parts of this neural network. You can imagine its thought process like a collection of finely tuned photoshop filters, strung together into a hierarchical pipeline. Naturally, the more complex stuff is at the end.

Source: http://jtnimoy.com/blogs/projects/50616707-deepdream-avoiding-kitsch

I wonder if this is only talking about the Deepdream filters from Dreamscopeapp, or all of the styles?

ProGamerGov commented 8 years ago

Here's a comparison album of CNNMRF with Dreamscope app: https://imgur.com/a/e24sj

There seem to be a few differences that almost look like I could change the settings to match in CNNMRF.

3DTOPO commented 8 years ago

I think the scaling is better fit for the Dreamscope examples - but other than that personally I think your CNNMRF results are much better.

ProGamerGov commented 8 years ago

@3DTOPO The "Triangle Style" in Dreamscope still looks better than CNNMRF: https://imgur.com/a/6ZWGJ Which is the main style I want to replicate.

dovanchan commented 7 years ago

So what do you guys thing about the dreamscope? where can we find the code that made the similar result with dreamscope? the most thing is that it just use 30s to make a own style result

ProGamerGov commented 7 years ago

@dovanchan As per one of Dreamscopeapp's owners, they are using a modified version of the "A Neural Algorithm of Artistic Style" research paper, so in theory it should be possible to recreate their results in Neural-Style with the appropriate modifications. The question is however what modifications are needed?

dovanchan commented 7 years ago

@ProGamerGov Hi,Can you share to us the best parameters you have that close to the dreamscope?

ProGamerGov commented 7 years ago

Researching things even more, it appears that Dreamscrope is related to a company called Lambda Labs, which caused a lot of controversy with their plans to produce facial recognition and surveillance software for the NYPD (New York Police?) using rooted Google Glass devices: https://angel.co/lambda-labs

This looks like their Github account: https://github.com/lambdal

They have someone related to OpenCV associated with the company. So maybe Dreamscope uses OpenCV? Stephen A. Balaban's Github account shows he was interested in OpenCV: https://github.com/stephenbalaban?tab=repositories

Some people here speculate that their DeepDream code is using something that creates similar results to DarkNet's DeepDream hallucinations?

ProGamerGov commented 7 years ago

Dreamscope's output:

My output:

The above images are from the results of my experiments with neural_style_laplacian.lua, for comparison with Dreamscope images:

https://imgur.com/a/Az8Y3

Using manual GIMP cropping and pasting, this is how close I was able to get to Dreamscope's results:

https://imgur.com/a/eH317


You can also compare my results on this issue, with the results from a model trained to better preserve facial features: https://github.com/jcjohnson/neural-style/issues/292#issuecomment-300685209