ROI for stream mode

[totaam]

Make the --encoding=stream mode (#3872) more appropriate for xpra by specifying unchanged screen regions. ffmpeg added support for this here: https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/418c954e318a79f77eae1b4d6b29d40daee4284a Original x264 commit is here (from 2012!): https://mailman.videolan.org/pipermail/x264-devel/2012-April/009327.html

Seems to involve:

• b_fast_pskip = 1: https://github.com/mirror/x264/blob/eaa68fad9e5d201d42fde51665f2d137ae96baf0/x264.h#L414
• mb_info: https://github.com/mirror/x264/blob/eaa68fad9e5d201d42fde51665f2d137ae96baf0/x264.h#L819-L839

When scaling server-side, the module will need to scale the ROI coordinates too.

---

Something similar should be possible with NVENC, unfortunately the documentation is seriously lacking.

int8_t                                     *qpDeltaMap;
/**< [in]: Specifies the pointer to signed byte array containing QP delta value per MB
in raster scan order in the current picture.
This QP modifier is applied on top of the QP chosen by rate control. */
uint32_t                                    qpDeltaMapSize;
/**< [in]: Specifies the size in bytes of qpDeltaMap surface allocated by client
and pointed to by NV_ENC_PIC_PARAMS::qpDeltaMap.
Surface (array) should be picWidthInMbs * picHeightInMbs */

---

libvpx: https://github.com/webmproject/libvpx/blob/main/examples/set_maps.c

[totaam]

For AMD: AMD Enables VCN Region Of Interest "ROI" Video Encoder Support For Linux GPU Drivers

[totaam]

With the ROI stub above, and add adding some debug logging to the x264 code:

           if( h->fdec->mb_info && (h->fdec->mb_info[h->mb.i_mb_xy]&X264_MBINFO_CONSTANT) )
            {
                d = h->fdec->i_frame - h->fref[0][0]->i_frame;
                qp = h->fref[0][0]->effective_qp[h->mb.i_mb_xy];
                maxqp = h->mb.i_qp;
                printf("xy=%d, d=%d, weighted_pred=%d, qp=%d - max=%d\n", h->mb.i_mb_xy, d, h->sh.b_weighted_pred, qp, maxqp);
                if( !SLICE_MBAFF && d == 1 && !h->sh.b_weighted_pred && qp <= maxqp )
                {

Not all odd macroblocks are being skipped as I would have expected - ie:

xy=3219, d=1, weighted_pred=0, qp=255 - max=19

Useful link for x264 defaults: https://github.com/cybertk/x264/blob/cf71aa426ac07a47b41f754b1d6b5d9e8fc39f5d/files/x264/common/common.c#L45

[totaam]

Findings:

• telling x264 to ignore some mbs is not enough, we have to ensure their contents really have not changed - failure to do so results in crashes! can we always guarantee that this is the case?
• the list of damage rectangles is being modified by the video-region handler and also when using "full-screen update because of too many regions" - I think we can safely ignore these?
• we may have run scroll detection by the time we call video-encode, can this be combined with the damage data to skip more areas? (remove these unchanged rectangles) we would need to be certain that the scroll-data corresponds to the last video frame! (this is hard)
• we will need to convert the rectangles to a list of mbs: probably best to add a rectangle.intersects(*rects) call and test for all mbs? is this expensive? or add a rects_intersect(*mb_rects1, *rects2) -> list[int] (list of mb indices)?