[macOS] StageWebView::drawViewPortToBitmapData() creates incorrect BitmapData with "high" "requestedDisplayResolution" and Retina screens

[itlancer]

Problem Description

StageWebView::drawViewPortToBitmapData() creates incorrect BitmapData (its content) with StageWebView(true) for macOS with Retina screens and <requestedDisplayResolution>high</requestedDisplayResolution> in application manifest. This issue blocks for us making "screenshot" of webframe viewport.

It has been tested with multiple AIR versions (even with latest AIR 33.1.1.633) with multiple macOS devices (Intel-based and ARM-based) with Retina screens with different OS versions with different AIR applications. Same problem in all cases. There is no such problem without Retina screens. Also without <requestedDisplayResolution>high</requestedDisplayResolution> in application manifest all works fine.

There is no such problem with Windows.

Steps to Reproduce

Launch code below with macOS device with Retina screen (application open website using StageWebView) and click anywhere on stage (red color) to make screenshot drawViewPortToBitmapData(). Screenshot (BitmapData) should be displayed right after StageWebView via Bitmap. Note: application manifest should contains <requestedDisplayResolution>high</requestedDisplayResolution>. Note: if you trying to launch application via Animate you will get incorrect BitmapData anyway, requestedDisplayResolution doesn't make sense.

Application example with sources attached. stagewebview_draw_viewport_to_bitmapdata_bug.zip

package {
    import flash.display.Sprite;
    import flash.events.Event;
    import flash.media.StageWebView;
    import flash.geom.Rectangle;
    import flash.events.MouseEvent;
    import flash.display.Bitmap;
    import flash.display.BitmapData;

    public class StageWebViewDrawViewPortToBitmapDataBug extends Sprite {
        private var stageWebView:StageWebView;

        public function StageWebViewDrawViewPortToBitmapDataBug() {
            addEventListener(Event.ADDED_TO_STAGE, init);
        }

        private function init(e:Event):void {
            removeEventListener(Event.ADDED_TO_STAGE, init);

            stageWebView = new StageWebView(true);
            stageWebView.loadURL("https://airsdk.harman.com");
            stageWebView.viewPort = new Rectangle(0, 0, 640, 480);
            stageWebView.stage = stage;

            stage.color = 0xff0000;
            stage.addEventListener(MouseEvent.CLICK, click);
        }

        private function click(e:MouseEvent):void {
            var bitmapData:BitmapData = new BitmapData(stageWebView.viewPort.width, stageWebView.viewPort.height, false);
            stageWebView.drawViewPortToBitmapData(bitmapData);//Draw incorrect content

            var bitmap:Bitmap = new Bitmap();
            bitmap.bitmapData = bitmapData;
            bitmap.width = stageWebView.viewPort.width;
            bitmap.height = stageWebView.viewPort.height;
            bitmap.x = 640;
            addChild(bitmap);
        }
    }
}

Actual Result: Screenshot (BitmapData) of webframe corrupted (incorrect):

Expected Result: Screenshot (BitmapData) of webframe correct (should looks like it shown inside StageWebView).

Known Workarounds

none *Don't use <requestedDisplayResolution>high</requestedDisplayResolution> StageWebView(false) and HTMLLoader uses outdated HTML specifications and doesn't work for ARM devices: https://github.com/airsdk/Adobe-Runtime-Support/issues/1106

[ajwfrost]

Hi

Just tried this using macOS 11.4 on M1, with AIR 33.1.1.633, and the bitmap that appeared on the right looked fine.. Do you know what exact versions of everything you were using to create the screenshot shown above? Not really sure what could the differences; the bitmap corruption looks a bit like it's a pixel format issue ..

thanks

[itlancer]

@ajwfrost Sorry for misunderstanding. I figure out. It happens only with <requestedDisplayResolution>high</requestedDisplayResolution> for macOS devices with Retina screens. Reproduced with MacBook Pro M1 macOS 11.6 and MacBook Pro A2141 16" 2019 macOS 11.3. Both with Retina display. With MacBook Air 2017 macOS 11.6 all works fine. Original issue description and application manifest (in attached zip) updated.

Testing sample with Animate always reproduce issue (no matter which value requestedDisplayResolution have) and it confused me.

[safason]

The workaround is functional, but removing requestedDisplayResolution on Mac OS X makes the text in app unreadable / blurred. It's quite critical here.

[safason]

Tested with AIR 33.1.1.731, on both Intel and M1 (Mac OS X Monterey v 12.0.1) the img captured is fuzzy. Without high resolution setting the fonts are unreadable.

[ajwfrost]

Hi @safason, @itlancer - have just got some feedback on this one: the problem with the malformed bitmap is purely because with the Retina display and high resolution mode, the drawing is happening at 2x the resolution so the 640x480 bitmap data is the wrong size: the area we get from the native WebView component is 1280x960. There's then a line-based bitmap copy happening which causes the interleaving effect that you see in the original image above.

Solutions: a) we can detect this scenario and look at scaling down the view into the smaller bitmap data so that the two images appear to be the same b) we could also allow you to pass in a bitmapdata object with 2x the width and the height of the viewport, so that you get the full resolution image and can then manipulate that however you wanted to...

thanks

[itlancer]

@ajwfrost, solution B with 2x BitmapData (by width/height) looks more acceptable cause we will get "full resolution" screenshot.

[safason]

B) seems good option

[ajwfrost]

It wasn't meant to be either/or :-)

So if you pass in a bitmapdata object with the same size as the viewport, you will get a valid image albeit scaled down. And if you pass in a bitmapdata object with 2x the width and 2x the height, you will get the image at the full scale provided by the OS. (If you try that on any runtime where it's not running in this mode, then you'll get the normal 'invalid parameter' error thrown as if you had passed in a randomly sized bitmapdata object...)

Hopefully then, if the higher resolution would be useful, you can try with 2x the width and height but catch the invalid parameter error in case it's not in the high-res mode, and then try again with width/height matching the viewport.

thanks

[itlancer]

It wasn't meant to be either/or :-)

So if you pass in a bitmapdata object with the same size as the viewport, you will get a valid image albeit scaled down. And if you pass in a bitmapdata object with 2x the width and 2x the height, you will get the image at the full scale provided by the OS.

@ajwfrost, that's even better :)

[safason]

@ajwfrost agree :)

[itlancer]

Fixed with latest AIR versions. Checked with latest AIR 33.1.1.856. Thanks!