--pa not working when using VBR mode

[GlitterKill]

I saw a note that --pa only works with VBR mode encodes but when I tested this with VBR I saw the following in the log output:

Pre-analysis is not supported on this device, disabled. VCEEnc (x64) 7.00 (r1066) by rigaya, Apr 30 2022 18:34:01 (VC 1931/Win) OS: Windows 11 x64 (22000) [UTF-8] CPU: AMD Ryzen 7 3700X 8-Core Processor [4.24GHz] (8C/16T) GPU: AMD Radeon RX 5700 XT, AMF Runtime 1.4.25 / SDK 1.4.24 Input Info: AviSynth+ 3.7.2 r3661(yv12)->p010 [AVX2], 1920x1080, 30000/1001 fps Vpp Filters copyHtoD cspconv(p010 -> yv12(16bit)) unsharp: radius 3, weight 0.5, threshold 10.0 cspconv(yv12(16bit) -> p010) Output: H.265/HEVC main @ Level 4 (high tier) 1920x1080p 0:0 29.970fps (30000/1001fps) Quality: slow VBR: 4000 kbps Max bitrate: 30000 kbps QP: Min: 0, Max: 51 VBV Bufsize: 30000 kbps Bframes: 0 frames Pre Analysis: off Motion Est: Q-pel Slices: 1 GOP Len: 300 frames Others: deblock vbaq

[rigaya]

I now have a test build which will allow using pre-analysis ( #54 ).

Please test with the test build uploaded at the links below.

• Win x86 link
• Win x64 link

[GlitterKill]

I downloaded the x64 build you linked and placed the files in staxrip/apps/encoders/vceenc (replacing existing files) but ran into an error when trying to do a VBR test encode. Here are the settings, error and log.

Log:

------------------------- System Environment -------------------------

StaxRip : v2.13.0 Windows : Windows 10 Pro 2009 Language : English (United States) CPU : AMD Ryzen 7 3700X 8-Core Processor GPU : Virtual Desktop Monitor, AMD Radeon RX 5700 XT Resolution : 3440 x 1440 DPI : 96 Code Page : 1252

----------------------- Media Info Source File -----------------------

C:\Users\Gary\Downloads\1080test.mp4

General Complete name : C:\Users\Gary\Downloads\1080test.mp4 Format : MPEG-4 Format profile : Base Media / Version 2 Codec ID : mp42 (mp42/mp41) File size : 2.63 GiB Duration : 59 min 3 s Overall bit rate mode : Variable Overall bit rate : 6 379 kb/s Encoded date : UTC 2017-02-10 11:21:47 Tagged date : UTC 2017-02-10 11:22:08 TIM : 00:00:00:00 TSC : 30000 TSZ : 1001

Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : Main@L4.1 Format settings : CABAC / 3 Ref Frames Format, CABAC : Yes Format, Reference frames : 3 frames Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 59 min 3 s Bit rate : 6 057 kb/s Width : 1 920 pixels Height : 1 080 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 29.970 (30000/1001) FPS Standard : NTSC Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.097 Stream size : 2.50 GiB (95%) Language : English Encoded date : UTC 2017-02-10 11:21:47 Tagged date : UTC 2017-02-10 11:21:47 Color range : Limited Color primaries : BT.709 Transfer characteristics : BT.709 Matrix coefficients : BT.709 Codec configuration box : avcC

Audio ID : 2 Format : AAC LC Format/Info : Advanced Audio Codec Low Complexity Codec ID : mp4a-40-2 Duration : 59 min 3 s Bit rate mode : Variable Bit rate : 317 kb/s Maximum bit rate : 417 kb/s Channel(s) : 2 channels Channel layout : L R Sampling rate : 48.0 kHz Frame rate : 46.875 FPS (1024 SPF) Compression mode : Lossy Stream size : 134 MiB (5%) Language : English Encoded date : UTC 2017-02-10 11:21:47 Tagged date : UTC 2017-02-10 11:21:47

----------------------------- Demux audio -----------------------------

MP4Box 2.1-DEV-rev79-gdf29bc8a0-x64-gcc11.2.0 Patman

C:\Temp\StaxRip\Apps\Support\MP4Box\MP4Box.exe -single 2 -out "C:\Users\Gary\Downloads\1080test_temp\ID1 English.m4a" C:\Users\Gary\Downloads\1080test.mp4

Start: 9:05:18 AM End: 9:05:19 AM Duration: 00:00:00

General Complete name : C:\Users\Gary\Downloads\1080test_temp\ID1 English.m4a Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom) File size : 135 MiB Duration : 59 min 3 s Overall bit rate mode : Variable Overall bit rate : 319 kb/s Encoded date : UTC 2022-07-18 14:05:18 Tagged date : UTC 2022-07-18 14:05:18

Audio ID : 2 Format : AAC LC Format/Info : Advanced Audio Codec Low Complexity Format profile : AAC@L2 Codec ID : mp4a-40-2 Duration : 59 min 3 s Bit rate mode : Variable Bit rate : 317 kb/s Nominal bit rate : 9 b/s Maximum bit rate : 325 kb/s Channel(s) : 2 channels Channel layout : L R Sampling rate : 48.0 kHz Frame rate : 46.875 FPS (1024 SPF) Compression mode : Lossy Stream size : 134 MiB (99%) Language : English Encoded date : UTC 2022-07-18 14:05:18 Tagged date : UTC 2022-07-18 14:05:19

---------------------------- Configuration ----------------------------

Template : H265 VBR Video Encoder Profile : AMD | H.265 VBR Container/Muxer Profile : MP4 (mp4box)

--------------------------- AviSynth Script ---------------------------

AddAutoloadDir("C:\Temp\StaxRip\Apps\FrameServer\AviSynth\plugins") LoadPlugin("C:\Temp\StaxRip\Apps\Plugins\Dual\L-SMASH-Works\LSMASHSource.dll") Import("C:\Temp\StaxRip\Apps\Plugins\AVS\MultiSharpen\MultiSharpen.avsi") SetFilterMTMode("DEFAULT_MT_MODE", 2) LSMASHVideoSource("C:\Users\Gary\Downloads\1080test.mp4") MultiSharpen(1) Prefetch(8)

------------------------- Source Script Info -------------------------

Width : 1920 Height : 1080 Frames : 106204 Time : 59:03.673 Framerate : 29.970029 (30000/1001) Format : YUV420P8

------------------------- Target Script Info -------------------------

Width : 1920 Height : 1080 Frames : 106204 Time : 59:03.673 Framerate : 29.970029 (30000/1001) Format : YUV420P8

---------------------- Media Info Audio Source 1 ----------------------

General Complete name : C:\Users\Gary\Downloads\1080test_temp\ID1 English.m4a Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom) File size : 135 MiB Duration : 59 min 3 s Overall bit rate mode : Variable Overall bit rate : 319 kb/s Encoded date : UTC 2022-07-18 14:05:18 Tagged date : UTC 2022-07-18 14:05:18

Audio ID : 2 Format : AAC LC Format/Info : Advanced Audio Codec Low Complexity Format profile : AAC@L2 Codec ID : mp4a-40-2 Duration : 59 min 3 s Bit rate mode : Variable Bit rate : 317 kb/s Nominal bit rate : 9 b/s Maximum bit rate : 325 kb/s Channel(s) : 2 channels Channel layout : L R Sampling rate : 48.0 kHz Frame rate : 46.875 FPS (1024 SPF) Compression mode : Lossy Stream size : 134 MiB (99%) Language : English Encoded date : UTC 2022-07-18 14:05:18 Tagged date : UTC 2022-07-18 14:05:19

--------------------------- Video encoding ---------------------------

VCEEnc 7.10

C:\Temp\StaxRip\Apps\Encoders\VCEEnc\VCEEncC64.exe --avsdll C:\Temp\StaxRip\Apps\FrameServer\AviSynth\AviSynth.dll --codec hevc --output-depth 10 --quality slow --tier high --pe --pa --pa-sc High --pa-ss High --pa-caq-strength High --colormatrix bt709 --colorprim bt709 --transfer bt709 --vbaq --vbr 4000 -i C:\Users\Gary\Downloads\1080test_temp\1080test_new.avs -o C:\Users\Gary\Downloads\1080test_temp\1080test_new_out.hevc

---

C:\Users\Gary\Downloads\1080test_temp\1080test_new_out.hevc

VCEEnc (x64) 7.01 (r1134) by rigaya, Jul 14 2022 12:57:39 (VC 1932/Win) OS: Windows 11 x64 (22000) [UTF-8] CPU: AMD Ryzen 7 3700X 8-Core Processor [4.25GHz] (8C/16T) GPU: AMD Radeon RX 5700 XT, AMF Runtime 1.4.25 / SDK 1.4.24 Input Info: AviSynth+ 3.7.2 r3661(yv12)->p010 [AVX2], 1920x1080, 30000/1001 fps Vpp Filters copyHtoD Output: H.265/HEVC main @ Level 4 (high tier) 1920x1080p 0:0 29.970fps (30000/1001fps) Quality: slow VBR: 4000 kbps Max bitrate: 30000 kbps QP: Min: 0, Max: 51 VBV Bufsize: 30000 kbps Bframes: 0 frames Pre Analysis: sc high, ss high, activity y, caq high Motion Est: Q-pel Slices: 1 GOP Len: 300 frames VUI: matrix:bt709,colorprim:bt709,transfer:bt709 Others: deblock pe vbaq Failed to send frame to encoder.

Start: 9:09:48 AM End: 9:09:54 AM Duration: 00:00:06

General Complete name : C:\Users\Gary\Downloads\1080test_temp\1080test_new_out.hevc File size : 0.00 Byte

------------------------- Error Muxing to MP4 -------------------------

Muxing to MP4 returned exit code: 1 (0x1)

---------------------------- Muxing to MP4 ----------------------------

MP4Box 2.1-DEV-rev79-gdf29bc8a0-x64-gcc11.2.0 Patman

C:\Temp\StaxRip\Apps\Support\MP4Box\MP4Box.exe -fps 29.970030 -add "C:\Users\Gary\Downloads\1080test_temp\1080test_new_out.hevc#video:name=" -add "C:\Users\Gary\Downloads\1080test_temp\ID1 English.m4a#audio:lang=eng:name=" -itags cover=C:\Users\Gary\Downloads\cover.png -new C:\Users\Gary\Downloads\1080test_new.mp4

[FileIn] Asked to read 5000 but got only 0 [FileIn] IO error EOF found after reading 0 bytes but file C:\Users\Gary\Downloads\1080test_temp\1080test_new_out.hevc size is 0 Cannot find track ID matching fragment #video Error importing C:\Users\Gary\Downloads\1080test_temp\1080test_new_out.hevc#video:name=: At least one required element has not been found

---------------------------- Muxing to MP4 ----------------------------

MP4Box 2.1-DEV-rev79-gdf29bc8a0-x64-gcc11.2.0 Patman

C:\Temp\StaxRip\Apps\Support\MP4Box\MP4Box.exe -fps 29.970030 -add "C:\Users\Gary\Downloads\1080test_temp\1080test_new_out.hevc#video:name=" -add "C:\Users\Gary\Downloads\1080test_temp\ID1 English.m4a#audio:lang=eng:name=" -itags cover=C:\Users\Gary\Downloads\cover.png -new C:\Users\Gary\Downloads\1080test_new.mp4

[FileIn] Asked to read 5000 but got only 0 [FileIn] IO error EOF found after reading 0 bytes but file C:\Users\Gary\Downloads\1080test_temp\1080test_new_out.hevc size is 0 Cannot find track ID matching fragment #video Error importing C:\Users\Gary\Downloads\1080test_temp\1080test_new_out.hevc#video:name=: At least one required element has not been found

Start: 9:09:54 AM End: 9:09:54 AM Duration: 00:00:00

[rigaya]

I've tested further for pre-analysis, it seems like pre-analysis seems to work only for H.264 encoding with RX5xxx series. (I've tested with RX5500XT). This might be hardware or driver limitation.

I'm not sure for RX6xxx series as I don't have one.

[arctus256]

I can confirm pre-analysis in HEVC works on RX6xxx series with (in particular RX6800).

Tested settings:

E:\video\test\test_temp\test_new_out.hevc

VCEEnc (x64) 7.01 (r1134) by rigaya, Jul 14 2022 12:57:39 (VC 1932/Win) OS: Windows 10 x64 (19044) [UTF-8] CPU: AMD Ryzen 7 5800X 8-Core Processor [4.85GHz] (8C/16T) GPU: AMD Radeon RX 6800, AMF Runtime 1.4.25 / SDK 1.4.24 Input Info: AviSynth+ 3.7.2 r3661(yv12)->nv12 [AVX2], 1912x1080, 24000/1001 fps Vpp Filters copyHtoD Output: H.265/HEVC main @ Level 4 (main tier) 1912x1080p 0:0 23.976fps (24000/1001fps) Quality: slow VBR: 8250 kbps Max bitrate: 12000 kbps QP: Min: 0, Max: 51 VBV Bufsize: 12000 kbps Bframes: 0 frames Pre Analysis: sc high, ss high, activity y, caq high Motion Est: Q-pel Slices: 1 GOP Len: 240 frames VUI: matrix:bt709,colorprim:bt709,transfer:bt709 Others: deblock pe vbaq

[rigaya]

@arctus256 Thank you for the information about RX6xxx series. Nice to hear it is supported for HEVC also in RX6xxx series.

So I think pre-analysis supports looks like below at this time. (I've checked RX460 and pre-analysis did not work for H.264)

Pre-analysis Support	GPU	H.264	HEVC
RX4xx, RX5xx series or before	no	no
RX5xxx series	yes	no
RX6xxx series	yes	yes

[rigaya]

VCEEnc 7.01 has the fix included for the pre-analysis issue, reflecting the test build change, I think I'll have this issue closed.

Thank you for reporting and having interest on this issue.