search

LibVNC / libvncserver

LibVNCServer/LibVNCClient are cross-platform C libraries that allow you to easily implement VNC server or client functionality in your program.
GNU General Public License v2.0
1.08k stars 483 forks source link

rfb.h cleanup + fix OOB memory accesses in tight encoder #534

Closed tobydox closed 1 year ago

tobydox commented 1 year ago

As of 7124b5fbcf0df8db4d3f73023d77af6ea56409e7 the maximum rect width and size are equal for every compression level so we can easily save the LUT entries by using constants instead.

This fixes OOB memory accesses in rfbNumCodedRectsTight() when being called by rfbSendFramebufferUpdate() before SendRectEncodingTight() is being called the first time (which limits cl->tightCompressLevel to the size of tightConf).

bk138 commented 1 year ago

Thanks so much for the nice contribution (again) @tobydox! I was seeing crashes in rfbNumCodedRectsTight() as well but couldn't quite figure out the reason.