How to choose / test best settings?

[N3tCode]

Hey there jackun,

First and foremost, thank you! Thank you, thank you, thank you! On to the questions.

I'm currently using dxtory with openencodevfw to record 1920 x 1080 30FPS video game footage. Knowing this, would you mind giving me advice on choosing the best openencodevfw settings for the game(s) I'm playing please?

I've been doing a lot of research and haven't found any concrete settings or explanations on when and what to use. What I've been going by thus far is the settings someone previously asked about, basically:

Mode 0 - QP 20 Quick Set - Quality Profile - High Level - 4.2

The thing is, I don't know why haha. You mentioned to someone else that QP 10 might be too much and QP 20 should be good. Well, how would one know what is enough? Is there anyway to benchmark our settings? All I know is when I turn the QP lower, the file size gets bigger (more lossless?).

Basically I've come down to this, I've been changing all these settings around and seeing how it impacts the performance of my game (basically watching the FPS in the top left and the writing FPS also in the top left). This has been hard to benchmark though.

My goal is this: Get the highest quality I possible can with the least amount of impact on my system. Any tips on how to benchmark or test this? I mean, why use lower quality settings if my computer can handle the higher quality ones?

I'm all over the place I'm sorry. I've spent the past week in evenings researching / working on all this.

One last thing, Compared to using the YUV420 BT 709 / YUV422 BT 709 vs the openencodevfw, I've noticed the color to be off a bit. It appears to be more red, but I'm not too sure (I remember you researching this). Do you know any way to fix this with sony vegas? Maybe something similar to the Computer RGB preset fix? I'd just rather not use the YUV since the file size is huge compared to using the openencodevfw codec. Much easier to manage yours :).

Thank you!!!

[jackun]

You can search info about x264 for applicable settings like QP (fixed quantization parameter). After trying few coefficients I think I settled on limited range Rec. 601 actually. I'll try to add rec. 709.

Benchmarking, hmm. Maybe avs2avi with something like so (edit encoder.avs): avs2avi encoder.avs out.avi -c h264 -w

but the reported fps seems to be lower than codec can do for me (using x264vfw for decoding, cpu usage 100% on one core hmm).

May need x264vfw for input file decoding if it too is h264.

With new DLLs you can run config. dialog with rundll32.exe OPENENCODEVFW.DLL,Configure

[N3tCode]

Thank you for the quick response.

I can't remember if I tested the 601 but I just remember the colors being spot on with 709 but when using openencodevfw the colors are off slightly. Is the encoding process causing this I suppose?

This is the post that seems to describe what I'm experiencing with the color: https://github.com/jackun/openencodevfw/issues/2

Also, do you mind answering what kind of difference the Profile and Level settings do? I read the wiki but it still doesn't explain it too well (at least not to me).

Thank you again.

[jackun]

MPC-HC has BT.601 to BT.709 shader, try that (with K-Lite codecpack anyway) though screenshotting with and without gives no difference. It poses some limitations on encoder like max bitrate, frame size etc so that low-powered devices can still playback the stream. But computers are not so limited so set it to maximum. Encoder/driver may override it with what it thinks is best anyway.

[N3tCode]

So if I understand this correctly,

If i use: Profile - High Level - 4.2

According to this compatibility chart: https://trac.ffmpeg.org/wiki/Encode/H.264#Compatibility

When I upload to youtube, an iphone 4 won't be able to access my video content?

Also, so the openencodevfw is based off of the x264 codec correct? So: https://trac.ffmpeg.org/wiki/Encode/H.264 Is relevant?

[jackun]

Youtube re-encodes anyway so don't sweat it.

[N3tCode]

Okay I THINK I'm starting to get it. It's nice actually having someone to talk to about this stuff. Plus, hopefully people will be able to reference this convo to reduce the question load as well.

With that being said, we're using H.264 right?

Youtube recommends: Video codec: H.264 Progressive scan (no interlacing) High Profile 2 consecutive B frames Closed GOP. GOP of half the frame rate. CABAC Variable bitrate. No bitrate limit required, though we offer recommended bit rates below for reference Chroma subsampling: 4:2:0

So If i use the Profile - High and Level - 4.2 at 1920 x 1080 that limits my max kbit/s to 62,500 correct? Well, what I can't find is what QP - 20 translates to bit wise? Any thoughts?

Oh and if youtube reincodes, and my goal is youtube, am I wasting my time / resources / space setting certain settings like the Level / quality / etc?

[jackun]

You can't find probably because QP sets "image quality" (kind of) so bitrate jumps around a lot. I've seen over 20Mbps with 1680x1050@30

[N3tCode]

Well that makes more sense.

Any thoughts on best settings for Youtube? That's really my only goal here.. I ask because the Youtube page doesn't make reference to anything about how it re encodes the Level setting for example.

I'm sorry jackun, I hope I'm not a thorn in your side.

[jackun]

If your upload and patience can take it then yeah why not upload QP 20 videos. Experiment :P You get lower quality streams after YT converts the video anyway so any device should be able to view it. And this vfw encoder uses AMD's older OpenEncode API and can't do B-frames anyway so no need to worry about that too.

http://www.pixeltools.com/rate_control_paper.html In H.264, each unit increase of QP lengthens the step size by 12% and reduces the bitrate by roughly 12%.

[N3tCode]

Okay I will continue the experiments! :) You've definitely helped a lot and I hope you know your work is greatly appreciated.

Two more things! Can I help in any way with the color investigation? Again if I use dxtory with YUV420 / YUV422 BT 709 it's a much larger file but the colors are spot on when compared to the openencodevfw where the colors are more red it seems.

And two, is there anyway we can donate to you?

[jackun]

1.) I'll look into it tomorrow and up new version 2.) no need :)

[N3tCode]

Thank you so much for looking into that. I'll be anxiously awaiting your update!

And no need? Why not boss?!

[jackun]

Ok, I added little something. Any difference?

BT.709 limited is currently a placeholder and actually uses "full range" or what wikipedia gives. BT.601 limited range gives values from 16..235.

E: Hmm, 24 bit rgb may develop green spots.

[N3tCode]

Good morning!

BT.601 limited range is the one that most closely matches the color, but it still seems to lose some of the green (in my video) and seems to replace it with a very light red (I believe it's red at least, the color is just off haha)

I was trying another codec and even this x264vfw codec makes good video where the color isn't off.

Thank you for investigating!

[jackun]

I can't tell the difference between this and x264vfw with limited BT.601. If you compare with yuv422 then ofcourse H264 yuv420 sampling is going to look worse. Other than that, BT.709 is too bright and a bit yellowish.

Compared to using the YUV420 BT 709 / YUV422 BT 709 vs the openencodevfw,

Where do you specify BT.709 for yuv exactly?

[N3tCode]

I'm using dxtory and was referring to the UtVideo YUV420 BT.709 VCM codec. But I get the same, good color results using another codec which is the x264vfw h.264/mpeg-4 codec as well.

I'm going to get you an example.. of what I'm seeing. / My entire process.

[N3tCode]

Both were re encoded with Sony .mp4 at 16mb and uploaded to youtube. YUV422.BT709 codec

openencodevfw

As you can see, the top picture blacks are a little more black / blue while the bottom picture the black s are replaced with a bluish / reddish tint.

But the thing is, even before sony and youtube, the top image looked just like the game where is the bottom was off. What I'm trying to say, is the sony / youtube process didn't affect the videos, they were always like this once dxtory finished recording them.

[jackun]

Dunno, I get the same color with this and x264vfw, that is, reds are equally messed up. Decoder difference? There used to be issue with red/blue because I mistakenly used red coefficients for blues and vice versa but that should be fixed now.

Source UtVideo bt709, OVE, x264 ...cut..

[N3tCode]

So you're thinking this is something that can't be fixed and I just need to live with? :)

[jackun]

Have you checked in plain media player like VLC or MPC-HC? Better yet post a source clips maybe. The thing is that HD stuff should be BT.709, but my tests are fine with BT.601 so something is screwy here.

[N3tCode]

At first I noticed an issue when I opened certain videos with Windows Media Player Vs. VLC, but for my current tests I added an extra step. I noticed a difference in my video player colors between the two codecs, but then I uploaded them both to Youtube, thus bypassing the video player's potential settings, and I still get the color differences.

The pictures taken above were both print screens why playing from from the youtube player.

[jackun]

If i re-encode the first image to huffyuv and run it through virtualdub, it gets yellowish for me: ove bt601 limited, x264

[N3tCode]

You're definitely more technical than me. I'm sorry for my lack of understanding. All I know is the process I use to record is different (I used YUV422.BT709 codec on top, and the openencodevfw on bottom) then re-encode with sony vegas in identical settings and get the following results as you see above.

But something to be noted is I get the same results prior to encoding with sony vegas or uploading to youtube. I just used youtube as a final test with a neutral player so to speak.

I'm sorta lost at this point on what to do. I'd much rather use your codec since it's using dedicated GPU hardware and provides less strain on my system compared to using software encoding ya know?

Are you trying to say that there shouldn't be a color difference between the two?

[jackun]

These are probably wrong coefficients but (if I uploaded the binaries) try that "BT.709 alt(ernate) full range" color matrix. Luma seems a bit brighter though.

E: Didn't upload... but try now. Edit edit: hmm, some coefficients need input RGB values to be in range of 16..235 instead of 0..255 like it is currently.

[N3tCode]

I'll have to try again in about 7 hours when I get home from work. I had tried them all before a couple days ago and remember the BT.601 limited range was the closest one, but I didn't upload to youtube. I was just testing with VLC and comparing them. I guess I can upload to youtube and see if the colors match up.

Is there a video player you recommend I use to test these videos? I ask because after if I try to open openencodevfw files with WMP, the colors are really off, but when I open them with VLC the colors are like they are when I upload them to youtube (I'm pretty sure at least). But I can't use VLC to open the UT YUV422.BT709 codec AVI files for some reason and have to use WMP, but this works because WMP produces perfect results when compared to my actually running game footage. Then I compare the running game footage to the uploaded youtube file from the YUV422.BT709 codec and it's still perfect. Though I should note that after I re encode with sony vegas, the YUV422.BT709 codec files will open with VLC and look identical to actual game footage / youtube upload footage.

Thank you for your help once again!

[jackun]

Ok, enough for today :P Fixed a typo and BT.709 full seems now to be most similar though a little blue shines through still.

[N3tCode]

Wow heck yeah man! It looks waaaaaayyyyyy better! Yep just a little blue it seems is coming in now with the BT.709 full.

What's helping 'fix' it?

The BT.709 full works best for me as well, with the BT.709 alt full coming in 2nd (it appears a little brighter all around).

But here's my issue now... If I render it with sony vegas, even though I choose the file I recorded with: the BT.709 full, once rendered, it returns the darker red (if I use the BT.601 Full range the colors are off through VLC but once I render it in sony vegas, it returns to the picture above). If I bypass sony vegas and just upload the BT.709 full .avi openencodevfw made straight to youtube, then it retains slightly bluish (yet good!) color.

What am I doing wrong? I really need the ability to render again or else I won't be able to add any further video editing you know? It does the same thing with Adobe premiere. Once I import it into either sony vegas or premiere, the video preview shows the colors incorrectly (so I know it'll come out even more incorrect when I render it again with my video editing changes).

It just doesn't make sense to my finite mind because if I switch codecs to the UT YUV422.BT709 or really any UT codec, the colors are perfect when they're first recorded and even once I render it through sony vegas it comes out compressed with still perfect colors... all the way to youtube. :(

Thank you.

[N3tCode]

Any thoughts?

[jackun]

Looks better. So it seems some floating point precision issue. Before I added 128 to UV values, but now it uses dot function (multiply/add). Oh well.

[N3tCode]

The link doesn't work.

And is 'Oh well' a good thing? :)

[jackun]

Try now. Also new bins should be up now. 'Oh well' as in 'Oh that floating-point!' lol.

E: Ooh, my flower pic's reds are also fixed with bt709 (source, x264).

If you test, bear in mind that the decoder should use the same color primaries/matrix whatever. BT.601 is used for SD and BT.709 for HD videos but some decoders are more accurate than others. See this doom9 post.

[N3tCode]

Thank you for that link, as well as continuing to help resolve my issue.

Can I ask you something though? Was there ever a color issue? There was right? I ask because I had a moment last night where I felt maybe there was no issue at all and it was just me heh...

Everything I record is HD though (seeing that my resolution is always 1920x1080) so if that's correct, according to the post, Sony Vegas will revert to BT.709. So in theory, the colors that I see with windows VLC should be the same that I see when placed into Sony Vegas correct?

[jackun]

Was there ever a color issue?

Yeah, using BT.601 for HD and some precision errors from separate multiplications/additions it seems.

VLC should be the same that I see when placed into Sony Vegas correct?

Hopefully, VLC/MPC-HC look the same to me anyway.

[N3tCode]

Thank you jackun. I've been driving myself a little crazy with all this testing and what not. Ya know, as you can probably tell, I don't really understand this all too well so you might be able to see how I've been second guessing myself ;).

Thank you again. I can't wait to get home (few hours) and test it out!

[N3tCode]

Using dxtory with the openencodevfw and the BT.709 full range recommended gives me (as much as my eyes can tell) a perfect match with what's being recorded when it creates the .avi file. The problem now is when I import this into sony vegas, I get the strong dark reds still :(.

The only way not to get the strong dark reds after I import to sony vegas is to use the BT.601 full range, but then I'm left with the differences from the picture of the two ships I posted above. - the colors are off. OR Not use the openencodevfw codec at all :( - which is obviously something I don't want to do.

According to the doom9 link, sony vegas should swap to bt.709 with HD (I'm assuming a 1920 x 1080 video is HD), but if so, why aren't the colors matching up? In theory, since I'm using openencodevfw and choosing bt.709 then recording at 1920 x 1080, shouldn't it import then render fine with sony vegas?

Please oh please help me. Thank you so much.

[jackun]

Hmm, mplayer plays with dark red too. I'll check how to modify H264 bitstream to tag it as BT.709.

E: Ok, set the bits but mplayer/vlc on linux are still off, weird. From MediaInfo:

Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Mplayer on linux BT.601 full. Blurrier but pretty much the same as source.

[N3tCode]

What I don't understand is according to doom9, VLC and Sony Vegas both ignore flags (tags) and use BT.601 or BT.709 depending on whether the source is SD or HD. It's strange that for me (windows) my VLC shows the BT.709 perfectly but when put into Sony Vegas it's incorrect. Would there be any way that a 1920 x 1080 video wouldn't be HD?

Also, if I remember correctly, Whenever I used the other codec YUV420 BT.601 it produced the same results as the codec YUV422 BT.709 (which is the codec I was using for the picture above since it reproduced the same colors are the actual source of my video).

Could it be that openencodevfw needs a color fix sorta like how you fixed the BT.709?

Lastly, I've noticed a pattern with doom9's findings with using BT.709 for HD things, why is that? Why not BT.601 for HD?

Thank you again boss!

[jackun]

Ok, added some code that sets some bits in H264 bitstream about colors (may be a bit crashy :P). Maybe Sony decoder now has a clue. Otherwise, I dunno, you could remux to mp4 or re-encode with x264 (easier with Handbrake, ffmpeg or MeGUI) and --crf 21 or something.

[N3tCode]

Okay so you want me to use the BT.709 full recommended still?

Also, is there a way to fix the BT.601 colors like you did for the 709?

[jackun]

See which one works.

BT.601 looks correct to me. Just on windows it seems to be decoded with BT.709 coeffs or...

[N3tCode]

Okay thank you. I'll try in the evening.

I just remember when I started all of this (before we had the ability to choose the BT #) the color was slightly off like you see in my original pictures above - which I now know was using BT.601 right?

[jackun]

Yep, but it probably had some precision errors.

[N3tCode]

My head is about to explode... I've been testing for hours heh,

But I was finally able to get the colors to match up so to speak (I think they match for the most part, they decrease in quality so I think that causes a slight color difference).

The only way I could figure out how to get the matching colors (for the most part) was to use openencodevfw BT.709 full recommended, then convert it to mp4 through handbrake, then import to sony vegas, then render it as mp4 again.

Good times. Any suggestions would be much appreciated because I don't think that this is the right process but maybe it is heh. I mean, is it bad to keep shrinking the file size, then expanding it again (avi -> mp4 (via handbrake - file size shrinks -> mp4 via sonyvegas - file size increases)?

Thank you.

[jackun]

Well, yes, you would want to have lossy steps as little as possible. I added another BT.601 coeffs I found that has YUV in range of 0..255 instead of 16..235. Fun indeed, on linux VLC displays BT.601 correctly but screenshotting makes it brighter. The new BT.601 stays the same when screenshot but plays darker and yellower on windows, bah.

[N3tCode]

I ran the NEW BT.601 through vegas and it appeared to come out like my ship pictures above (light red tint). When I opened it with VLC it was definitely darker, but vegas does weird things to files.

[N3tCode]

And hey jackun, when it comes to video editing, should I just stick with software codecs if I want the good color AND want to edit the renderings?

I mean if I just wanted a perfect .avi file of my game recording and didn't want to add anything to it through premiere / vegas then heck yeah I would stay with openencodevfw because the changes you did to BT.709 are spot on once they finish recording.

But is it too much to ask from the x264 codec to get the colors right and work with video editing programs or is there some 3rd (4th / 5th?) program I should be using instead of premiere / vegas?

I'm just shooting for the most practical process for someone who works full time, is married, and somehow doing game videos on the side as a hobby.

I appreciate all your work and don't doubt your skills, I'm just starting to doubt x264 (compared to those UT YUV420/YUV422 BT.709 codecs).

OH and I was going to look into the Bandicam AMD App codec, have you used it at all? Is it their own developed codec?

[jackun]

One thing, are you adding AVIs straight to Vegas? If Bt.709 is screwed up, I think that might be it. Vegas is then probably using x264 to decode and x264 screws up colors. Vegas 13 and BT.709 full remuxed to MP4 and render with Sony AVC/Internet profile (which looks ok on windows but dark red on linux, oh boy lol).

See issue #7 some info about remuxing.

E: Meh, uploaded bunch of videos to youtube. BT.601, while yellower in Vegas, looks the most like source pic to me. Maybe cause linux flash is still at version 11 (edit: yep). edit edit: BT.709 looks fine on windows' flash player :P

[N3tCode]

You're a genius! Yes I was just going straight from dxtory recording .avi to vegas. But I 'remuxed' (wasn't familiar with this term) to mp4 and the colors line up!

The only thing now is the audio tracks don't remux with it. That normal? I've just been taking the original .avi from dxtory and adding the audio to vegas normally than deleting the video and transferring the new mp4 version.

And I 'think' the last thing I'm confused about is when I upload to youtube, the colors go from that dark red to normal colors when I change from 360p to 1080p quality using the youtube viewer option. - This doesn't seem normal. I tested it on other people's videos and I don't see their's doing it, maybe I'm not looking close enough?... Any thoughts?

Thank you so much again!

[N3tCode]

Yeah that my mp4box gui does't work for me and I can't find any documentation on how to install it or whether not I need to use the mp4box with it as well. I got another program to work MkvtoMp4, seems to work fine but my audio codecs aren't showing wavelengths..

[jackun]

Another wall of text about colors. But it might just need levels(1) adjustments(2). BT.709 360p/480p looks same to me as 1080p with windows' flash and HW acceleration enabled (software rendering, accelerated decoding), hmmm. Maybe HW accel. makes a difference. brain -> :boom: