Does kvazaar support SHVC encoding?

Wilson-Ian commented 4 years ago

From the web site I only found two encoders who claimed they have the capability to support SHVC (HEVC scalable extension). one is ATEME, and one is Allegro DVT.

Just to confirm here with expert whether kvazaar support it or any plan to support it?

Thanks very much!

Wilson

fador commented 4 years ago

Hi,

Yes we support SHVC, we have a separate Scalable branch for that. @MrAsura could tell more on what we actually support..

-Marko

MrAsura commented 4 years ago

Currently the Scalable branch has mostly been tested with the ultrafast preset / settings close to the ultrafast preset, so there are propably tool combinations that don't work properly.

Stuff that is supported (or at least has been fairly well tested):

2x/1.5x and SNR scalability with --gop=0 (other scaling ratios should work but not tested)
for paralellism tools: threads, owf, and wpp mostly work (some occasional checksum mismatches still seem to happen randomly)
for compression tools: tmvp, deblock, and SAO should work with --gop=0

Stuff that has been somewhat tested but might not completely work:

B-frame pyramid gop (--gop=8) and low delay P-frame gop should work at least if bi-pred is disabled (enabling tmvp may cause hash mismatches)
Tiles may somewhat work when the latests commits get pushed to here

There probably won't be much more support/bug fixes to scalable kvazaar unless theres more demand for it.

Wilson-Ian commented 4 years ago

Thanks very much @MrAsura @fador for your detail information! From Scalable Branch doc I can see some parameters are introduced for layers such as --layer, --layer-res, --input-layer etc.

I tried the function by the following command and got a bit stream with 2 layers. kvazaar.exe -i s.yuv --input-res 1920x1080 --layer -o s.hevc

1) What's the parameter to apply SNR scalability as you helped mention in previous post? Thanks!

2) Would it be possible to use command to encode/generate output stream file per layer? or Is there any existing tools we can use to extract the output bitstream with multiple layers to separated stream so that one stream represented one layer? or we have to use openHEVC, for example, to extract / decode / merge by self? Please help advise. Thanks again!

MrAsura commented 4 years ago

The documentation for the Scalable branch may not be fully up-to-date/clear, but hopefully a few usage examples will make things more clear.

2 layer video with separate input files with 2x spatial scaling: kvazaar.exe -i <input1>_960x540.yuv --preset=ultrafast --gop=0 --layer -i <input2>_1920x1080.yuv --preset=ultrafast -o bin.hevc
2 layer video with one input file and 2x spatial scaling: kvazaar.exe -i <input1>_1920x1080.yuv --preset=ultrafast --gop=0 --layer-res=960x540 --layer --preset=ultrafast --gop=0 --input-layer=0 --layer-res=1920x1080 -o bin.hevc
2 layer video with SNR (quality scalability): kvazaar.exe -i <input1>_1920x1080.yuv --preset=ultrafast --gop=0 -q 34 --layer --preset=ultrafast --gop=0 --input-layer=0 -q 28 -o bin.hevc

The --layer parameter is used to add a new layer and any parameters after it modify the new layers parameters. The --layer-res <resolution> parameter is used to specify the resolution that should be used for the current layer if it differs from the input resolution (the input is automatically scaled to match the layer resolution if they differ). --input-layer <ind> is used to specify the input (files specified with the -i parameter) used by the current layer. (--input-res <resolution> needs to be used if the input files resolutions is not in the file name)

As for the other questions:

Temporal layers are enabled if --gop=t8 is specified. Both spatial and quality layers are supported (see usage examples).
I am not aware of any analysis software that has support for the Scalability Extension for HEVC and I doubt it would be possible to analyze SHVC bitstream (or atleast the higher layers) without explicit support for SHVC. openHEVC should have support for SHVC but I haven't used it, so I don't know if it is enabled by default or a separate branch etc. The reference encoder/decoder SHM is another option for decoding SHVC bitstream.
As far as I'm aware, only one output file/bitstream is generated when encoding SHVC. At least this is the default behaviour of the reference encoder SHM. Althoug, it should be possible to output separate streams with some modifications, but at least scalable kvazaar doesn't support it. For sending SHVC bitstream from a server etc., it is my understanding that a separate device/program would be used to parse the NALU headers and only send the NALUs with layer id 0 for BL or send the whole bitstream for BL+EL. Having two files from the start would probably work too, but syncronization might be a problem.

Hopefully this cleared things up a bit more!

Wilson-Ian commented 4 years ago

Appreciate your help @MrAsura!

It's time to study your code now :)

Wilson-Ian commented 4 years ago

Hi @MrAsura,

Sorry to bother again! I tried the SHVC with two layers with 2X spatial scaling and encoding result is impressive!! For EL bits, it reduced almost 47% and BL bits is also reduced 27% as well, comparing with standalone HD and LD hevc stream.

While I use SHM12.4 TAppDecoder the SHVC stream I got an error about the checksum, and only the EL stream is succsesfully decoded to yuv file and can be played by YUVPlayer or ffplay.

But for BL, the output file size is zero. I understood currently the kvavzaar project might not be your priority, would it be possible to provide any hints or suggestions, so that I can dig into the code a little efficiently. Thanks very much! The detail info that I have is below:

use kvz to encode shvc spatial scaling stream: kvazaar.exe -i BasketballDrive_960x540_25.yuv --preset=ultrafast --gop=0 --layer -i BasketballDrive_1920x1080_25.yuv --preset=ultrafast -o kvz_Basketball2x_25.hevc

Each yuv has 501 frames.

use shm12.4 decoder to decode it TAppDecoder.exe -ls 2 -b kvz_Basketball2x_25.hevc -o0 kvz_Basketball2xBL.yuv -o1 kvz_Basketball2xEL.yuv

Result: the kvz_Basketball2xEL.yuv is good. but the file size of kvz_Basketball2xBL.yuv is zero. And got checksum error for 68 out of 1002 frames during the decoding. The interesting point is even thought the checksum error is detected are all for layer1 (LId: 1) but the layer1 stream can be outputted correctly!

POC 2 LId: 1 TId: 0 ( P-SLICE TRAIL_R, QP 25 ) [DT 0.065] [L0 2(0, {2.00, 2.00}x)c 1 ] [L1 ] [Checksum:0fa1f351,03f50767,03e877a7,(***ERROR***)] [rxChecksum:0fa34957,03f4f7e2,03e86f47] POC 3 LId: 0 TId: 0 ( P-SLICE TRAIL_R, QP 26 ) [DT 0.013] [L0 2c ] [L1 ] [Checksum:03e49962,00fa5a4c,0107f12b,(OK)] POC 3 LId: 1 TId: 0 ( P-SLICE TRAIL_R, QP 26 ) [DT 0.059] [L0 3(0, {2.00, 2.00}x)c 2 ] [L1 ] [Checksum:0fa2de30,03f51a60,03ea5cf5,(***ERROR***)] [rxChecksum:0fa3da18,03f505a5,03ea599a] POC 4 LId: 0 TId: 0 ( P-SLICE TRAIL_R, QP 23 ) [DT 0.030] [L0 3c ] [L1 ] [Checksum:03e63f14,00fa20cb,010881e4,(OK)] POC 4 LId: 1 TId: 0 ( P-SLICE TRAIL_R, QP 23 ) [DT 0.115] [L0 4(0, {2.00, 2.00}x)c 3 ] [L1 ] [Checksum:0fa02eda,03f54964,03ec33b7,(***ERROR***)] [rxChecksum:0fa0f7e2,03f52e80,03ec3dfb] POC 5 LId: 0 TId: 0 ( P-SLICE TRAIL_R, QP 26 ) [DT 0.016] [L0 4c ] [L1 ] [Checksum:03e71b2d,00fa2ba0,01087b79,(OK)] POC 5 LId: 1 TId: 0 ( P-SLICE TRAIL_R, QP 26 ) [DT 0.068] [L0 5(0, {2.00, 2.00}x)c 4 ] [L1 ] [Checksum:0f9c1530,03f3f290,03edc423,(***ERROR***)] [rxChecksum:0f9cb80a,03f3e1de,03edd08c] POC 6 LId: 0 TId: 0 ( P-SLICE TRAIL_R, QP 25 ) [DT 0.014] [L0 5c ] [L1 ] [Checksum:03e75f30,00fa6b17,0108b1fb,(OK)]

Wilson-Ian commented 4 years ago

I got the solution. My usage of SHM is not correct. now I can extract the BL YUV succesfully. Please ignore above post and Merry Christmas!

malleshamdasari commented 3 years ago

@MrAsura Thanks for all insights. I see that Kvazaar supports only two layers max. Is it just Kvazaar or SHVC too?

MrAsura commented 3 years ago

The SHVC standard does allow for more than two layers, so with SHM it should be possible to include more layers.

The Scalable Kvazaar architecture would allow for more than two layers, but there might still be hard-coded values etc. in the parameter sets that would cause problems when decoding, if more than two layers are used.

malleshamdasari commented 3 years ago

I see. I used Kvazaar to encode two layers it worked fine. But, it was throwing an error when I tried with more than 2 layers. I will take a look at SHVC if it allows more layers with no issues. Thanks.

aviadzuc commented 2 months ago

I got the solution. My usage of SHM is not correct. now I can extract the BL YUV succesfully. Please ignore above post and Merry Christmas!

I got the solution. My usage of SHM is not correct. now I can extract the BL YUV succesfully. Please ignore above post and Merry Christmas!

Hi, I know it's been a long time, but I'm running into the same issue and can't figure out what I did wrong - scalable kvazaar encoding of two-layer file, decoding using SHM 12.4 and getting one null file back. If there's any chance you can post your solution I'd appreciate it

ultravideo / kvazaar

Does kvazaar support SHVC encoding? #254