search

satoshiiizuka / siggraphasia2019_remastering

Code for the paper "DeepRemaster: Temporal Source-Reference Attention Networks for Comprehensive Video Enhancement". http://iizuka.cs.tsukuba.ac.jp/projects/remastering/
Other
486 stars 103 forks source link

Incomplete coloring with larger --mindim values. #31

Open kepppis opened 2 years ago

kepppis commented 2 years ago

I've run couple times into the situation where the coloring is perfect if mindim is less than 450, but with larger values the colorization will fade and the output is with pale colors which are very far from reference images. Tested this with two environments, one with Nvidia Tesla k40 with 11 GB ram and pytorch 1.2.0 (the last version compatible with oldish Tesla card).
Other with Nvidia Quadro P5000 with 16 GB ram and pytorch 1.8.1. The source video and the reference pics are 576p and the only thing I am changing is the mindim value.

kepppis commented 2 years ago

Here are sample videos of the problem - same source and same reference pictures:

https://youtu.be/Qfq6L0l4dk8 --mindim 540

https://youtu.be/OjyTnmFb_PY --mindim 320

The red color is the always the first to disappear and fade.

vitacon commented 1 year ago

@kepppis Have you managed to solve this? It's a shame it seems the authors lost insterest in this project.

kepppis commented 1 year ago

Unfortunately not, but I've found a way to get pretty similar results to highres coloring. I do the coloring on lower resolution and overlay it over the highres bw image using suitable color blending filters.

vitacon commented 1 year ago

@kepppis Yeah, that's a classic trick. =) Unfortunately the "color mask" is still too blurry to my taste, color details are lost {eyes, lips, etc.} and colors are bleeding.

017819

I wondered if it would be possible to force a higher working resolution and maybe retrain the model but I guess I'll start from a different place.

kepppis commented 1 year ago

Let me know if you find anything usable.

vitacon commented 1 year ago

Well, I tried EbSynth for that and I use my own tool Tinterr (eriador.condak.cz/tools.php), but to get a decent result you have to feed it with a LOT of keyframes...

vitacon commented 1 year ago

My test result (about 1560 frames and 23 key frames) can be seen here: https://www.youtube.com/watch?v=D9bcxx4xL00

None of this methods is perfect...

kepppis commented 11 months ago

Is it possible to have instructions how to use PNG-Tinterr ?

vitacon commented 11 months ago

Is it possible to have instructions how to use PNG-Tinterr ?

Yes, but I have to create a provisional English manual first... I should have created it a long time ago. =]

vitacon commented 11 months ago

@kepppis Well, here is something that might help you: http://eriador.condak.cz/tools/png-tinterr-manual-eng.htm