High bit rate hevc still has some mosaics in some frames

[leokinglong]

1. Encode command： SvtHevcEncApp -i 2_hd.yuv -w 7680 -h 3840 -fps 30 -intra-period 4 -irefresh-type 0 -vid-info 1 -rc 1 -tbr 62914560 -umv 0 -tile_slice_mode 1 -tile_row_cnt 12 -tile_col_cnt 12 -pred-struct 0 -b 2_hd.hevc

2.Problems : Some of the pictures have some mosaics（especially on the face），how to fix this problem？

Thanks!

[tianjunwork]

Hi @LeoKingLong , what is the actual bitrate of the bitstream? You need to encode large enough number of frames to get a stable output bitrate. You can use -n number_of_frames to repeatedly encode a clip. If you still see block artifact on the face, could you attach the image?

[leokinglong]

Hi @tianjunwork The original video（mp4 format） information is here：

I used ffmpeg command to make a yuv from the original MP4： ffmpeg -y -i 1.mp4 -vf scale=7680:3840,setsar=1:1 1.yuv

Then I encoded the 1.yuv with SVT.

The image is here: https://leokinglong.github.io/output7.png https://leokinglong.github.io/output9.png

[tianjunwork]

Hi @LeoKingLong , what is the output bitrate from SVT-HEVC? You can find it in the log print after encoding. The output is very blocky, not only on face. Could you try with higher bitrate? The bitrate is cut half from original AVC bitrate. Actually, HEVC offers from 25% to 50% better data compression at the same level of video quality compared to AVC. Most of the time it is not exactly 50%.

[leokinglong]

Hi @tianjunwork The output bitrate from SVT-HEVC is 48725.12kbps

Then I make a new hevc :

It looks better , thanks

[leokinglong]

Hi @tianjunwork I found the parammeter -segment-ov-file ，but I don't know how to set the param in mode 3... 3. Use this mode to modify not only quality but also blocks splitting, behaviour depends on parameter 1 and 2

What's the 'behaviour depends on parameter 1 and 2' meaning? How to set param in mode 3, can you give me some example? Thanks

[tianjunwork]

Hi @tszumski , Sorry it has been long after you implemented this. I also have difficulty to understand below statements.

1. Could you provide an example how to use -segment-ov-file and what is the purpose of using it. Is this a standalone feature?
2. Could you help to answer @LeoKingLong 's questions?
3. Could you make a PR by adding more explanation in this file SVTSegmentOvFile.txt? Thank you.

#====================== Segment Override File ======================
# One input frame have to be preceded by the number of regions
# Each region consist of 6 numbers (X1,Y1,X2,Y2, data, mode) i.e (X1,Y1)top left and (X2,Y2)bottom right corner
# Mode parameter have to be in range [1..5]. This allow user to specify what to override by the [data]
#    1.   Set qp value for each region, [data] is clipped to [0:51]
#    2.   Change qp value i.e increase or decrease by the [data] clipped to [-25:25]
#    3.   Use this mode to modify not only quality but also blocks splitting, behaviour depends on parameter 1 and 2
#    4.   Use this mode to modify blocks splitting i.e increase or decrease by the [data] clipped to [-25:25], negative value increase block division
#    5.   Update first stage of quantization in range [-7;7] positive value reduce bitrate, negative increase

[intelmark]

Also description in user guide (https://github.com/OpenVisualCloud/SVT-HEVC/blob/master/Docs/svt-hevc_encoder_user_guide.md) could use more detail/clarity. Could a use case description be added (e.g. provide more quality/less compression to a region of interest) be added as well? Explain what the difference would be between modifying qp vs modifying the block splitting?

[tszumski]

Hi @LeoKingLong, @tianjunwork @intelmark

1. What's the 'behaviour depends on parameter 1 and 2' meaning? It means that depend if you choose option 1. "Set qp value for each region" or 2. "Change qp value" combined with option 3. you will override block splitting by 1. "setting new qp value" or 2. "increase/decrease qp value"
2. How to set param in mode 3, can you give me some example? To set param in mode 3 you you have to add two lines, first in mode 1 or 2 and second in mode 3 as in example below:

   0 0 500 500 35 1 //set new qp value
   0 0 500 500 35 3 // modify also block splitting
   or 
   0 0 500 500 -5 2 //increase block quality by 5
   0 0 500 500 -5 3 // modify also block splitting

3. What is the purpose of using it This feature is rather for very specific use-case like surveillance systems, where objects like pedestrians or vehicles are detected and tracked (encoded with higher quality) while background is encoded with worse quality. Thanks to encoding only key areas with good quality bitstream is smaller and still have high value
4. Could you make a PR by adding more explanation in this file SVTSegmentOvFile.txt? Yes, I will do it when everything will be clear

[tianjunwork]

@LeoKingLong, according to @tszumski, this feature is very specific to the use case that the area where you want to get finer control of the quality is fixed. It can't be used in generic scene.

Thank you @tszumski for the explanation. So mode 1,2,4,5 can be used alone, right? Only mode 3 depends on already-configed mode 1 or 2. I think it is ok to add more details just in SVTSegmentOvFile.txt instead of user guide. Could you please make a PR that adds:

1. Purpose/use case of this feature
2. Provide more explanation and example to each modes. e.g. what we discussed above
3. Maybe change 1. to mode 1. 1. seems like step 1

Thank you.

[leokinglong]

@tianjunwork @tszumski

I tested mode 2 combined with mode 4, and I find a bug :

1. SVTSegmentOvFile.txt is set like this:

   2
   0 0 500 500 -15 2
   0 0 500 500 -20 4

2. The encode command is ： SvtHevcEncApp -i 2_hd.yuv -w 7680 -h 3840 -fps 25 -intra-period 4 -irefresh-type 0 -vid-info 1 -rc 1 -tbr 83886080 -umv 0 -tile_slice_mode 1 -tile_row_cnt 12 -tile_col_cnt 12 -pred-struct 0 -segment-ov-file 2-4.txt -b 2_hd.hevc

3. Bug: (1) Most frames of the generated hevc looks fine , but few frames quality is poor , the person in the picture is splited into some small blocks and the picture is corrupted. (2) I found out the corrupted frame and change the line in SVTSegmentOvFile.txt：

   1
   0 0 500 500 13 1

   The new hevc file look nice and no corrupted frame any more.

So... mode 2 is not allowed to combined with mode 4？

Later I will try mode 1 combined with mode 4 and feedback.

[tianjunwork]

@LeoKingLong Thanks for sharing the experiment. From the explanation, it seems only mode 3 can be combined with mode 1 or 2. @tszumski could you confirm?

[tszumski]

@LeoKingLong, @tianjunwork All modes are interchangeable 1 & 3, 1 & 4, 2 & 3, 2 & 4, @LeoKingLong could you share corrupted image or at least corrupted area? Then I could investigate this behavior.

[leokinglong]

@tszumski en....It seems like residual picture .....

[tszumski]

Thanks @LeoKingLong for images

For me it looks like expected behavior(you are adjusting quality quite much "-20"), what I would suggest is to set similar values both for mode 2 and 4, for example:

2
0 0 500 500 -15 2
0 0 500 500 -15 4

Then there should be less corruption, but please keep in mind that at the boundary of adjusted area there will be visible quality change. To soften that you can create more segments, for example:

6
0 500 500 700 -7 2
0 500 500 700 -7 4
500 0 700 700 -7 2
500 0 700 700 -7 4
0 0 500 500 -15 2 
0 0 500 500 -15 4

Or simplified version:

4
0 0 700 700 -7 2
0 0 700 700 -7 4
0 0 500 500 -15 2 
0 0 500 500 -15 4

[tianjunwork]

Hi @LeoKingLong , #555 is merged to explain more about -segment-ov feature. Are there any other topics you would like to discuss with this issue? If not, would you please close the issue? Thank you.

[leokinglong]

Thanks~@tianjunwork