servo / surfman

Accelerated offscreen graphics for WebGL
Apache License 2.0
172 stars 84 forks source link

fix(macos): add reference count to layer to prevent use after free #304

Closed wusyong closed 1 month ago

wusyong commented 1 month ago

On macOS, winit will try to release all related resources after a window is dropped. And surfman's macos system surface also contains wrappers of CALayer for its layer and superlayer field which will also try to free when dropping. If either winit window or Surfman surface decides to drop, it will cause use after free to each other.

If winit window is dropped first, surfman will get:

* thread #1, name = 'main', queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x8)
  * frame #0: 0x000000019ea66990 QuartzCore`CA::Layer::update_removed_sublayer(CA::Transaction*, unsigned int) + 60
    frame #1: 0x000000019ea60bec QuartzCore`CA::Layer::update_sublayers(CA::Transaction*, CALayerArray*, CALayerArray*) + 700
    frame #2: 0x000000019ea83130 QuartzCore`CA::Layer::destroy() + 124
    frame #3: 0x000000019ea8304c QuartzCore`-[CALayer dealloc] + 96

If surfman surface is dropped first, winit window will get:

* thread #1, name = 'main', queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x34)
  * frame #0: 0x000000019ec9d0c4 QuartzCore`CA::Layer::mark_visible(CA::Transaction*, bool, bool) + 28
    frame #1: 0x000000019ec9d22c QuartzCore`CA::Layer::mark_visible(CA::Transaction*, bool, bool) + 388
    frame #2: 0x000000019ea669bc QuartzCore`CA::Layer::update_removed_sublayer(CA::Transaction*, unsigned int) + 104
    frame #3: 0x000000019ea66848 QuartzCore`CA::Layer::remove_sublayer(CA::Transaction*, CALayer*) + 136
    frame #4: 0x000000019ea58eb0 QuartzCore`CA::Layer::remove_from_superlayer() + 172
    frame #5: 0x000000019a0d44e8 AppKit`-[NSView _removeLayerFromSuperlayer] + 252
    frame #6: 0x000000019a0d3ef0 AppKit`-[NSView _setSuperview:] + 280
    frame #7: 0x000000019a0f40c8 AppKit`-[NSView removeFromSuperview] + 156
    frame #8: 0x000000019a152a44 AppKit`-[NSView removeFromSuperviewWithoutNeedingDisplay] + 44
    frame #9: 0x000000019a0f8800 AppKit`-[NSView _finalize] + 680
    frame #10: 0x000000019a0f8440 AppKit`-[NSView dealloc] + 128
    frame #11: 0x000000019a331bb4 AppKit`-[NSFrameView dealloc] + 164
    frame #12: 0x000000019a331b00 AppKit`-[NSTitledFrame dealloc] + 72
    frame #13: 0x000000019a331a7c AppKit`-[NSThemeFrame dealloc] + 624
    frame #14: 0x0000000198082118 Foundation`_NSKVOPerformWithDeallocatingObservable + 172

This PR tries to add a reference count when creating the surface. This should set the correct reference count for both of them to drop.