manuelruder / artistic-videos

Torch implementation for the paper "Artistic style transfer for videos"
Other
1.75k stars 252 forks source link

Flickering Background? #59

Open ProGamerGov opened 6 years ago

ProGamerGov commented 6 years ago

I'm using the default artistic-videos specific parameters from stylizeVideo.sh, and a custom set of Neural-Style parameters. I suspect the default temporal coherence related parameters are to blame?

ProGamerGov commented 6 years ago

So it seems that my very amateur mistake of forgetting the seed value parameter, and not using a high enough temporal weight, were the cause of the issue.

But now I am noticing the non moving background gets darker over time, and this makes it hard to create a looping video/gif. How can I fix this drift on the non changing static background?

ProGamerGov commented 6 years ago

Using temporal_weight=1e4 combined with -num_passes 5 -use_temporalLoss_after 1:

Pass 1: https://gfycat.com/HastyBackBabirusa

Pass 2: https://gfycat.com/UnkemptAnxiousAgouti

Pass 3: https://gfycat.com/SmartDisloyalAntlion

Pass 4: https://gfycat.com/SilentImaginaryIguana

Pass 5: https://gfycat.com/ScaryDemandingAlpineroadguidetigerbeetle

It looks like I just needed to perform more passes.

However, a really high temporal weight (100,000) will cause some weird artifacts:

Pass 1: https://gfycat.com/WildEagerHadrosaurus

Pass 2: https://gfycat.com/UniformImpeccableLcont

rahat-yasir commented 6 years ago

How did you solve the temporal coherence issue ? Deep flow estimation takes hours for videos, did you use deep flow or, normal optical flow ?

ProGamerGov commented 6 years ago

@rahat-yasir My issue was with making a looping gif. The solution for that was just more passes. But some people told me that you can also split the video at different points every so often, so that the start and end are at different points, and this can help make them temporally coherent.

I haven't messed around with anything more than 30-60 frames, so I wouldn't know where the start for larger videos.