manuelruder / artistic-videos

Torch implementation for the paper "Artistic style transfer for videos"
Other
1.75k stars 252 forks source link

Conversion stopping after a few frames #6

Closed ghost closed 8 years ago

ghost commented 8 years ago

The optical flow has been successfully computed with Deep Matching and Deep Flow. However when proceeding to stylifying the images, the process stops at the 3rd or 4th. Any thoughts ?

Iteration 500 / 1000 Content 1 loss: 1703792.031250 Temporal 1 loss: 45211.532593 Style 1 loss: 2419.038391 Style 2 loss: 23474.081421 Style 3 loss: 28717.443848 Style 4 loss: 399992.993164 Style 5 loss: 3270.827103 Total loss: 2206877.947769 Iteration 600 / 1000 Content 1 loss: 1703150.468750 Temporal 1 loss: 45308.925629 Style 1 loss: 2418.774605 Style 2 loss: 23459.86785 Style 3 loss: 28724.108887 Style 4 loss: 400028.320312 Style 5 loss: 3272.040176 Total loss: 2206362.506218

relative change in function value is less than tolFunRelative Iteration 651 / 1000 Content 1 loss: 1702996.406250 Temporal 1 loss: 45350.559235 Style 1 loss: 2418.964767 Style 2 loss: 23457.32879 Style 3 loss: 28720.361328 Style 4 loss: 399955.883789 Style 5 loss: 3271.706390 Total loss: 2206171.210556 avconv version 9.18-6:9.18-0ubuntu0.14.04.1, Copyright (c) 2000-2014 the Libav developers built on Mar 16 2015 13:19:10 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1) Input #0, image2, from 'video20/out-%d.png': Duration: 00:00:00.20, start: 0.000000, bitrate: N/A Stream #0.0: Video: png, rgb24, 350x250, 25 fps, 25 tbr, 25 tbn [libx264 @ 0x1143140] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2 [libx264 @ 0x1143140] profile High, level 1.3 [libx264 @ 0x1143140] 264 - core 142 r2389 956c8d8 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.25 aq=1:1.00 Output #0, mp4, to 'video20-stylized.mp4': Metadata: encoder : Lavf54.20.4 Stream #0.0: Video: libx264, yuv420p, 350x250, q=-1--1, 25 tbn, 25 tbc Stream mapping: Stream #0:0 -> #0:0 (png -> libx264) Press ctrl-c to stop encoding frame= 5 fps= 0 q=32763.0 Lsize= 28kB time=0.12 bitrate=1880.1kbits/s video:0kB audio:0kB global headers:0kB muxing overhead 72210.256410% [libx264 @ 0x1143140] frame I:1 Avg QP:27.64 size: 12578 [libx264 @ 0x1143140] frame P:2 Avg QP:29.01 size: 5289 [libx264 @ 0x1143140] frame B:2 Avg QP:30.24 size: 1711 [libx264 @ 0x1143140] consecutive B-frames: 20.0% 80.0% 0.0% 0.0% [libx264 @ 0x1143140] mb I I16..4: 2.0% 54.5% 43.5% [libx264 @ 0x1143140] mb P I16..4: 1.7% 4.3% 2.6% P16..4: 40.2% 34.2% 15.8% 0.0% 0.0% skip: 1.3% [libx264 @ 0x1143140] mb B I16..4: 0.0% 0.0% 0.4% B16..8: 45.5% 17.8% 4.0% direct: 5.8% skip:26.6% L0:28.4% L1:42.6% BI:29.0% [libx264 @ 0x1143140] 8x8 transform intra:53.5% inter:55.1% [libx264 @ 0x1143140] coded y,uvDC,uvAC intra: 92.2% 100.0% 97.3% inter: 45.6% 63.3% 19.0% [libx264 @ 0x1143140] i16 v,h,dc,p: 0% 53% 0% 47% [libx264 @ 0x1143140] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 15% 27% 14% 5% 6% 6% 10% 6% 11% [libx264 @ 0x1143140] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 15% 25% 15% 8% 8% 7% 9% 7% 8% [libx264 @ 0x1143140] i8c dc,h,v,p: 47% 22% 14% 18% [libx264 @ 0x1143140] Weighted P-Frames: Y:50.0% UV:50.0% [libx264 @ 0x1143140] ref P L0: 83.5% 16.1% 0.4% [libx264 @ 0x1143140] kb/s:1063.12
manuelruder commented 8 years ago

In an earlier version there was a bug that the process stopped after 5 frames, if you don't specify "-num_images" explicitly. However, it should autodetect the number of frames with the latest version. Have you tried updating? If you already have the latest version, there should be an output "Detected n content images." What is it showing?

ghost commented 8 years ago

Problem solved after pull. Thx!