Lag on iPad when sim is zoomed in

[KatieWoe]

Test device iPad Air 2 Operating System iPadOS 15.5 Browser Safari Problem description For https://github.com/phetsims/qa/issues/804. Did not seem to happen on Win 11 Chrome or on other sims on the iPad. Zooming in to the sim seems laggy and panning, with or with out objects, while zoomed in also lags. The sim itself does not seem to lag.

Visuals

https://user-images.githubusercontent.com/41024075/169864148-834fc9bd-38c6-4571-86b8-93ea1e597035.MOV

[KatieWoe]

Ok, I have no idea why everything in the video is green. That definitely didn't happen while testing. The file plays fine on my computer, so something happened while uploading to github it seems.

[KatieWoe]

Here's a screen recording of the video. Uploading the video to Slack produced the same problem.

[pixelzoom]

This seems like something that should be addressed for FB 1.2 release. And it seems like a general problem that may affect all sims.

@KatieWoe are other sims also "laggy", or is this specific to Function Builder?

@jessegreenberg thoughts on what could be causing this, and how to address it?

[KatieWoe]

Did not seem to happen ... on other sims on the iPad.

I only checked one or two published sims, but it seemed specific to function builder/basics.

[pixelzoom]

@KatieWoe said:

Ok, I have no idea why everything in the video is green ...

It looks fine to me. Perhaps it's a playback problem on your machine.

[pixelzoom]

I cannot reproduce this on iPad6 + iPadOS 15.5 + Safari. Everything runs smoothly.

@KatieWoe can you please test master, or a recently-published dev version? If it's a recent regression, it's unlikely to be present in published sims. And since this feature is not at all sim-specific, I can't imagine why it would occur in Function Builder but not in other sims.

[Nancy-Salpepi]

Side note: @KatieWoe it isn't your machine. The video is green when I watch it on the Dell (win10 + chrome) but fine when I watch it on my mac with chrome.

[jessegreenberg]

I confirmed the performance problem on my iPad Air 2. I found that performance of pan and zoom is significantly better if the workaround for https://github.com/phetsims/function-builder/issues/69 is removed.

const WORKAROUND_35_OPTIONS = platform.mobileSafari ? { renderer: 'canvas' } : {};
const controlsLayer = new Node( WORKAROUND_35_OPTIONS );

Is it possible to remove that now?

[pixelzoom]

I could investigate removing the workround. But perhaps we should investigate why pan/zoom performance is so poor with renderer: 'canvas'. There are other sims that rely on canvas, and will presumably have this same problem. @jessegreenberg @jonathanolson thoughts?

[jessegreenberg]

Checking out some other usages: renderer: 'canvas' is used on the skater in energy-skate-park and performance was good there on my iPad Air 2. wave-interference uses it and framerate was slower there, but acceptable in my opinion. Performance was OK for large CanvasNodes like the ones in rutherford-scattering.

I am not sure why it is so poor for the controlsLayer, @jonathanolson may know if there is anything at the scenery level that can be done.

[pixelzoom]

I don't know if it's significant, but https://github.com/phetsims/scenery/issues/1424 (screenshot problem) was also reported on iPad Air 2 by @KatieWoe, and I could not reproduce on iPad6. Is there something that both Screenshot and Pan/Zoom are using that is having problems on iPad Air 2?

[pixelzoom]

Until I hear otherwise, labeling this as blocking. @amanda-phet please weigh in.

[jonathanolson]

It sounds like the screenshot and pan/zoom issues are both Canvas-related on the iPad Air 2? I couldn't reproduce slowness on my laptop (with the Canvas workaround applied). I don't have an iPad recent enough to test or reproduce otherwise. Setting the entire sim or screenview to Canvas could possibly improve performance too if the workaround is needed. There were two Canvases needed to display things, and more SVG layers:

If the iPad is compositing-limited, reducing the number of those layers could help.

[jonathanolson]

I was looking into something else (https://github.com/phetsims/function-builder/issues/145), and laptop Safari slowed down to a crawl on ?rootRenderer=canvas. The bulk of the time is spent applying a Canvas filter (ColorMatrixFilter of some type) with the per-pixel operations and get/put of data. This is possibly related.

[jonathanolson]

We're left with this slow option, since Safari does not implement Canvas filters (https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/filter) - so Canvas on Safari will be quite negatively impacted when there is an active filter, especially one that is having to filter something on a screen-size Canvas.

[pixelzoom]

@jonathanolson This problem occurs on iPad Air 2, but is not reproducible on iPad6 or desktop. Performance is fine on iPad6 and Safari desktop -- not slow at all. So I'd like to know how you came to this conclusion that "we're left with this slow option" for Safari.

[pixelzoom]

Since I don't feel like we're moving forward on any kind of general solution, I decided to try removing the phetsims/function-builder#69 workaround.

@KatieWoe please test 1.2.0-dev.8. Does it resolve the problem?

[amanda-phet]

@pixelzoom said

Until I hear otherwise, labeling this as blocking. @amanda-phet please weigh in.

I agree that this should be addressed, if possible, since this is important for all sims.

[pixelzoom]

I asked @jonathanolson:

... So I'd like to know how you came to this conclusion that "we're left with this slow option" for Safari.

We discussed on Zoom, and I think I understand now. The combination of renderer:canvas and filters (anything from scenery/js/filters) causes poor performance on Safari. ButtonNode (the base class of all sun buttons) uses 2 filters. So developers must really be careful about where they use renderer:canvas. In the case of the workaround for phetsims/function-builder#35, renderer:canvas was indeed being applied to a Node who had subclasses of ButtonNode as children (ResetAllButton and EraserButton specifically).

Alterting developer about this bad combination is tricky, because the Screenshot feature uses the same codepath. And the Screenshot feature relies on both canvas and filters. This explains the poor performance of Screenshot on Safari in phetsims/scenery#1424.

I created https://github.com/phetsims/scenery/issues/1417 to address the general issue.

[KatieWoe]

Looks much better now :)

[pixelzoom]

To verify in 1.2.0-rc.1, confirm that performance is acceptable when using the pan/zoom feature on iPadOS 15.5. Please test on an iPad Air 2, and on an iPad.

[KatieWoe]

This looks good in rc.1. Closing