VP9 high bit depth encoder crash

GoogleCodeExporter commented 9 years ago

What is the expected behavior? What do you see instead?
This bug was reported by Brendan Bolles(brendan@fnordware.com).

vpxenc is not crashing, so I wonder if I have a problem with how I'm 
configuring the encoder.  I don't get the same crash with 8-bit, just 10-bit 
and 12-bit.

What version are you using? On what operating system?

Can you reproduce using the vpxdec or vpxenc tools? What command line are
you using?

Please provide any additional information below.

Original issue reported on code.google.com by yunqingw...@google.com on 2 Feb 2015 at 9:56

GoogleCodeExporter commented 9 years ago

Brendan,
Could you provide some details - the platform, the configuration you used, and 
vpxenc command line? I will try to reproduce the crash.

Original comment by yunqingw...@google.com on 2 Feb 2015 at 9:59

GoogleCodeExporter commented 9 years ago

I used a commit from earlier today, 80e729f6018d9af2733bc45d91442391484a202b.

I built the config headers with ./configure --enable-vp9-highbitdepth.  I then 
copy them to a folder used by Xcode.

I'm building using Xcode 3.2.6 on OS X 10.6.8. The compiler is the system 
default, GCC 4.2.  These are what my config settings look like inside the 
plug-in:

    g_usage                      = 0
    g_threads                    = 8
    g_profile                    = 2
    g_w                          = 160
    g_h                          = 90
    g_bit_depth                  = 10
    g_input_bit_depth            = 10
    g_timebase.num               = 1
    g_timebase.den               = 24
    g_error_resilient            = 0
    g_pass                       = 0
    g_lag_in_frames              = 25
    rc_dropframe_thresh          = 0
    rc_resize_allowed            = 0
    rc_scaled_width              = 0
    rc_scaled_height             = 0
    rc_resize_up_thresh          = 60
    rc_resize_down_thresh        = 30
    rc_end_usage                 = 0
    rc_target_bitrate            = 256
    rc_min_quantizer             = 0
    rc_max_quantizer             = 63
    rc_undershoot_pct            = 100
    rc_overshoot_pct             = 100
    rc_buf_sz                    = 6000
    rc_buf_initial_sz            = 4000
    rc_buf_optimal_sz            = 5000
    rc_2pass_vbr_bias_pct        = 50
    rc_2pass_vbr_minsection_pct  = 0
    rc_2pass_vbr_maxsection_pct  = 2000
    kf_mode                      = 1
    kf_min_dist                  = 0
    kf_max_dist                  = 9999

Original comment by fnordw...@gmail.com on 2 Feb 2015 at 10:26

GoogleCodeExporter commented 9 years ago

would you mind trying git# f6c2a6c5? and let us know how it goes?

We are investigate a separate bug in commit# be6aeada, affecting highbitdepth.

Original comment by ya...@google.com on 2 Feb 2015 at 10:30

GoogleCodeExporter commented 9 years ago

Interestingly (or not), if I set the frame size to 160x90 like 
park_joy_90p_10_420.y4m, I get a crash in vp9_get16x16var_sse2() during the 
second pass.

If I use a frame size of 256x107 I get a crash in vp9_diamond_search_sad_c() 
during the first pass.

Original comment by fnordw...@gmail.com on 2 Feb 2015 at 10:32

GoogleCodeExporter commented 9 years ago

Tried f6c2a6c5, same result.

Original comment by fnordw...@gmail.com on 2 Feb 2015 at 10:51

GoogleCodeExporter commented 9 years ago

The issue occurs while building for a 32bit target. The following functions 
have bugs:
vp9_highbd_subpel_variance.asm
vp9_highbd_sad4d_sse2.asm

Work on it now to get a fix done.

Original comment by yunqingw...@google.com on 4 Feb 2015 at 7:51

Changed state: Started

GoogleCodeExporter commented 9 years ago

My plug-in is 64-bit only. Hope those fixes will apply to me as well.

Original comment by fnordw...@gmail.com on 4 Feb 2015 at 7:57

GoogleCodeExporter commented 9 years ago

What I can see were bugs in 32bit code. Anyway, please try the attached patch.

Original comment by yunqingw...@google.com on 4 Feb 2015 at 8:15

Attachments:

patch.diff

GoogleCodeExporter commented 9 years ago

Original comment by yunqingw...@google.com on 4 Feb 2015 at 8:17

GoogleCodeExporter commented 9 years ago

BTW, I went to see if I would get similar problems on Windows, but I wasn't 
able to build on the command line.

./configure --target=x86_64-win64-vs9 --enable-static-msvcrt 
--enable-vp9-highbitdepth
make

I don't get the errors if --enable-vp9-highbitdepth is left off.

Original comment by fnordw...@gmail.com on 5 Feb 2015 at 12:56

GoogleCodeExporter commented 9 years ago

Yes, the bugs are in high bit depth functions.

I tried to build 64bit libvpx on Mac and Linux. Both worked fine. The 
configuration used on Mac:
../libvpx/configure --target=x86_64-darwin10-gcc --enable-vp9-highbitdepth
then ran make, and ran vpxenc to encode the file.

Did you try the patch?

Original comment by yunqingw...@google.com on 5 Feb 2015 at 1:08

GoogleCodeExporter commented 9 years ago

I tried the patch, got the same crashes in my plug-in. Haven't been able to 
reproduce on the command line.

Original comment by fnordw...@gmail.com on 5 Feb 2015 at 5:08

GoogleCodeExporter commented 9 years ago

Ok. Then we may have bugs somewhere else. Could you build libvpx and run vpxenc 
to test if the problem exists in libvpx or not?

I assume you build libvpx on mac. Could you add the target explicitly? Run 
configuration:
$ ./configure --target=x86_64-darwin10-gcc --enable-vp9-highbitdepth

Then, run 
$ make

Then, encode a clip:
$./vpxenc --codec=vp9 -o out.webm ./park_joy_90p_10_420.y4m -p 2 --good 
--cpu-used=3 --target-bitrate=100 --auto-alt-ref=1 --fps=25/1 
--minsection-pct=0 --maxsection-pct=2000 --lag-in-frames=25 --kf-min-dist=0 
--kf-max-dist=125 --static-thresh=0 --min-q=0 --max-q=63 --arnr-maxframes=7 
--arnr-strength=5 --arnr-type=3 --profile=2 --bit-depth=10

Let me know if you see any problem.

Original comment by yunqingw...@google.com on 5 Feb 2015 at 5:33

GoogleCodeExporter commented 9 years ago

I've already said that it works on the command line with vpxenc. It still does, 
running the command you gave me. I don't know why the plug-in has problems that 
vpxenc doesn't.

I'm assuming that when you use --enable-vp9-highbitdepth, the only difference 
is in the various config files:

vp8_rtcd.h
vp9_rtcd.h
vpx_config.asm
vpx_config.c
vpx_config.h
vpx_scale_rtcd.h
vpx_version.h

Original comment by fnordw...@gmail.com on 5 Feb 2015 at 6:11

GoogleCodeExporter commented 9 years ago

Another test I just tried is to link to the library built by make on the 
command line instead of Xcode which I usually use. Drum roll…same result.

Meanwhile, --enable-vp9-highbitdepth doesn't build at all on my Windows config, 
as I said a few posts up.

My guess is it has something to do with how I'm configuring the encoder or how 
I'm passing it frames.

Original comment by fnordw...@gmail.com on 5 Feb 2015 at 6:27

GoogleCodeExporter commented 9 years ago

Original comment by fgalli...@google.com on 5 Feb 2015 at 7:06

Added labels: Hotlist-IndianRunnerDuck

GoogleCodeExporter commented 9 years ago

The configure problem on windows seems to be caused by some problem with 
doxygen. I tried a work around: 
./configure --target=x86_64-win64-vs9 --enable-static-msvcrt 
--enable-vp9-highbitdepth --disable-docs --disable-install-docs 

Then highbitdepth  builds fine.

Original comment by ya...@google.com on 5 Feb 2015 at 7:41

GoogleCodeExporter commented 9 years ago

Issue 935 has been merged into this issue.

Original comment by ya...@google.com on 5 Feb 2015 at 8:18

GoogleCodeExporter commented 9 years ago

OK, I figured out what my main problem was: not using the 
VPX_CODEC_USE_HIGHBITDEPTH flag when calling vpx_codec_enc_init(). Is that flag 
going to be there permanently?  Seems like when I set config.g_bit_depth = 10, 
the encoder would know to do VPX_CODEC_USE_HIGHBITDEPTH.

So that fixes the crashing on Mac. On Windows I'm getting a crash on 
vpx_codec_destroy().

Original comment by fnordw...@gmail.com on 5 Feb 2015 at 11:35

GoogleCodeExporter commented 9 years ago

BTW, to build on Windows with ./configure --target=x86_64-win64-vs9 
--enable-static-msvcrt --enable-vp9-highbitdepth I had to make this change to 
line 39 of vpx/vpx_integer.h

#define INT64_MAX _I64_MAX
+#define INT32_MAX _I32_MAX
+#define INT32_MIN _I32_MIN
#define INT16_MAX _I16_MAX
#define INT16_MIN _I16_MIN

Original comment by fnordw...@gmail.com on 5 Feb 2015 at 11:38

GoogleCodeExporter commented 9 years ago

I've been able to duplicate my Windows crash on the command line. Maybe this 
should be a new issue.

./configure --target=x86_64-win64-vs9 --enable-static-msvcrt 
--enable-vp9-highbitdepth
make
vpxenc --codec=vp9 --bit-depth=10 --profile=2 -o ./outfile.webm 
./infile_256x107_10bit_420.y4m

I don't get the crashes when I test park_joy_90p_10_420.y4m, I think because 
it's too short. The file I'm testing is 72 frames and it gets that same crash 
on vpx_codec_destroy().

Original comment by fnordw...@gmail.com on 6 Feb 2015 at 11:14

GoogleCodeExporter commented 9 years ago

Can you share the source video with us? for repro your crash. Thanks

Original comment by ya...@google.com on 6 Feb 2015 at 5:16

GoogleCodeExporter commented 9 years ago

Here you go.

Original comment by fnordw...@gmail.com on 6 Feb 2015 at 5:30

Attachments:

tears_of_steel_256x107_10bit_420.y4m

GoogleCodeExporter commented 9 years ago

thanks for the file, we were able to track down the issue with a fix: 
https://gerrit.chromium.org/gerrit/#/c/73703/1

Original comment by ya...@chromium.org on 6 Feb 2015 at 8:24

GoogleCodeExporter commented 9 years ago

Great, I'll try it.

Can you also make the #define INT32_MAX change I mentioned above?

Original comment by fnordw...@gmail.com on 6 Feb 2015 at 8:37

GoogleCodeExporter commented 9 years ago

Yep, it stopped my crash, good job!

Will push out a new beta when it makes it into the repo.

Original comment by fnordw...@gmail.com on 6 Feb 2015 at 9:19

GoogleCodeExporter commented 9 years ago

This should probably be a seperate issue, but somewhere along the line a 
thread-related bug appeared when encoding VP8.

./configure --target=x86_64-win64-vs9 --enable-static-msvcrt 
--enable-vp9-highbitdepth
make
vpxenc --threads=4 -o ./outfile.webm ./park_joy_90p_8_420.y4m

I get a crash with this. Not sure how the state of my repo stacks up against 
whatever you're using.

Original comment by fnordw...@gmail.com on 6 Feb 2015 at 10:54

GoogleCodeExporter commented 9 years ago

https://gerrit.chromium.org/gerrit/#/c/73707/ should fix vp8 crash. 

we clearly never tested vp8 in config where highbitdepth is enabled.

Original comment by ya...@google.com on 7 Feb 2015 at 12:36

GoogleCodeExporter commented 9 years ago

Yep, that fixed it. Thanks!

I take it --enable-vp9-highbitdepth will eventually go away and just be part of 
the default?

Original comment by fnordw...@gmail.com on 7 Feb 2015 at 3:24

GoogleCodeExporter commented 9 years ago

Original comment by fgalli...@google.com on 9 Feb 2015 at 10:42

Changed state: Fixed

reimaginemedia / webm

VP9 high bit depth encoder crash #940