Performance not acceptable on iPad

[jbphet]

Performance on iPad 3 is quite poor. Using version 1.0.0-dev.4, the speed and smoothness of the molecules in the container on the first two screens is noticeably laggy and chunky. When the ?profiler query option was used, the readout indicated that the sim was reaching about 29 fps when neon was put into the gaseous state and no other interaction occurred. When the heater/cooler button was then used to alternatively heat or cool the gas at about a 2 second cycle (1 second of heating followed by 1 second of cooling, then repeat), the frame rate dropped to around 13 fps.

This is gonna need some optimizing to achieve an acceptable level of performance.

[jbphet]

The two devices that I'm using to measure performance on iPads are Tycho, which is an iPad 2 model A1395 running iOS 9.3.3, and Disessa, which is an iPad 3 model A1416 running 9.3.3. The performance issues are the most apparent when displaying water, which makes sense, since both the model and the view are more complicated.

I'm using one general test with three options to benchmark the performance. The test is to load the simulation with the profiler query parameter, go to the first screen, select either Neon, Oxygen, or Water in the "Atoms & Molecules" control, then select "Gas" as the phase, and record the FPS value at 5, 15, and 30 seconds.

I did some quick testing at the beginning of the day today and was seeing about 9 fps on Tycho and 7 fps on Disessa for the worst case test, which is water, using the current master version. Here are some benchmarks after the set of optimizations done today:

Test	FPS on Tyco	FPS on Disessa
Neon gas @ 5 sec	30	27
Neon gas @ 15 sec	25	30
Neon gas @ 25 sec	30	29
Oxygen gas @ 5 sec	29	27
Oxygen gas @ 15 sec	30	30
Oxygen gas @ 25 sec	29	30
Water gas @ 5 sec	12	11
Water gas @ 15 sec	15	11
Water gas @ 25 sec	15	14

The values of 30 fps look pretty decent, and I think we could live with that performance. However, the frame rates for water are in the 11-15 fps range, and it definitely looks pretty bad to my eye, so there is still a ways to go on this.

I didn't take formal measurements, but when I looked at water on Disessa this morning, I was seeing around 7 fps, so the work done today improved the performance by 60-100%.

[jbphet]

For the record, water in the gaseous state looks great and runs at 60 fps on Leibniz, an iPad 4 model A1566 running iOS 9.3.2.

[jbphet]

I found an article on line where it said that calls to context.fill() and context.stroke() are expensive so it's best to try to minimize them. I tried restructuring ParticleCanvasNode.paintCanvas to do all the drawing, then call fill and stroke. When profiled on Chrome, it seemed to bring about a slight performance improvement, but we're already getting 60 fps easily on Chrome on my machine. When tried on the iPad 2 and iPad 3 used for the testing shown above, performance actually decreased. I guess the article was wrong, and it is perhaps better to do a bunch of little fills and strokes than a few big ones.

[jbphet]

At this point, after the optimizations done so far, performance looks pretty good for everything except water on the "Phase Changes" screen. For this particular setup, the frame rate is around 10-15 fps.

[jbphet]

I added some temporary code to collect data about how long it takes to run the model and to do the rendering on iPads. This code collects the processing time for 100 iterations of the model and 100 instances of rendering for water on the "Phase Changes" screen. Screenshots are shown below.

Based on this data, it would appear that the model iterations are taking up significantly more time than the rendering. The model iterations are taking on average 51.88 ms whereas the rendering is taking on average 6.91 ms, so the model step is using ~7.5 times as much as the rendering. Seems like I should focus my efforts on the model, and not think about looking at doing a WebGL node.

[jbphet]

While collecting this data, I found that the model often starts out with a relatively long execution for the very first time through, then does a number of iterations that take on the order of 50ms, then starts running at around 19ms per iteration. Below is an example of this happening (I put a little red scribble where the interaction time dropped down).

At this point, I have no idea why the times drop down like this. After a number of tests, it appears that when it happens is not deterministic, meaning that sometimes it happens within 100 iterations of the model, sometimes it happens after 200 iterations. This makes me think that it is some sort of optimization kicking in or something.

[jbphet]

Water on the "Phase Changes" screen is now acting very similar to water on the "States" screen. Turns out that there was an optimization for the latter that I'd just missed applying to the former. That has been corrected.

[jbphet]

Another optimization issue that I'll need to look at is that the performance goes down when the lid assembly (i.e. the lid, the pressure gauge, the thermometer, and the hand) is moving on the 2nd screen. This applies both to when it is being manually moved by the user or motion is being animated after explosion of the container. The performance drop is visible in the speed at which the atoms/molecules move.

To duplicate:

• run http://www.colorado.edu/physics/phet/dev/html/states-of-matter/1.0.0-dev.11/states-of-matter_en.html?profiler on an iPad 2 (notice the use of the profiler query parameter)
• select the "Phase Changes" screen (screen 2)
• note the FPS value shown by the profiler information - on the iPad 2 labeled "Tycho" this value generally settles in at 30 FPS
• while watching the motion of the atoms (which should be neon), start moving the lid up and down without going far enough down to interact with the atoms
• note that the rate at which the atoms visibly slows
• note the FPS value after a few seconds of continuously moving the lid up and down - on Tycho it gets down to around 8 FPS

[jbphet]

As mentioned in the previous comment, the frame rate drops noticeably on iPad2 when the lid is being moved up and down. I added some temporary code to measure how much of the change was due to the model being stepped and how much of it was due to the display being updated. The code was added (temporarily) to Sim.js in the stepSimulation method and made use of the function performance.now() to obtain high resolution timestamps and determine the amount of time for model+view stepping versus the call to updateDisplay. I gathered 300 deltas for each and averaged them.

Ultimately, what a learned was that more time is being taken up in the stepping than in the update display. Here is a summary of the data:

Test	Avg Time (ms)
model+view step, lid still	6.57
model+view step, lid moving	32.21
updateDisplay, lid still	6.15
updateDisplay, lid moving	13.63

So, when the lid is still, the step and the display update take roughly the same amount of time. When the lid is moving, the step time increases by a factor of almost 5, while the display update time increases by a factor of a little more than 2. Improving both would help, but obviously there is more to be gained by working on the model+view step time.

When I looked at this in Chrome dev tools profiling, it seemed like it was the setting of the node positions resulting from the movement in the model that was taking a lot of the time. A possible approach to improve performance might therefore be to simplify the bounds calculations for these nodes.

[jonathanolson]

Was that profiling done in master, or on the performance branch? I've done optimization on the performance branch that should improve the CPU usage during the model+view step.

[jbphet]

I've integrated the performance improvements from the performance branch into master, so the optimizations are reflected in the data above.

[jbphet]

I added a preventFit flag to the ParticleContainerNode, made several measurements on iPad 2, and saw a 23.6% reduction in the time for updateDisplay and a 14.3% reduction in the time for stepping the model. These numbers are based on the average of 50 data points for each measurement.

I compared this to having the preventFit flag set for all the subcomponents of this node, i.e. the pointing hand, the lid, the thermometer, and the pressure gauge, but saw only about 1/2 as much improvement.

I'm not sure that I entirely trust these numbers since there often seems to be a fair amount of variance in these measurements on an iPad, but having said that, it does seem like the lid moves a little more smoothly and the particles don't slow down quite as much. So, bottom line: The flag has been added.

[phet-steele]

Performance is still not ideal, but @jbphet recollects a recent meeting where @ariel-phet approved of the current performance. @ariel-phet, would you mind taking one last look, for posterity, at 1.0.0-rc.1 and confirm that the performance is what you expect?

[ariel-phet]

@phet-steele please comment on what you find "unideal" so I can look at those things specifically.

[phet-steele]

1. The momentary lag when switching to water for the first time
2. The particles slowing down when the lid is being moved by the user (any molecule)
3. General decrease in performance when things begin to heat up w/ large quantities of a molecule (especially water)

[ariel-phet]

@phet-steele do these problems exist on iPad Air? or just on iPad 2 for you?

[phet-steele]

@ariel-phet just iPad 2, iOS 9.3.5

[ariel-phet]

@phet-steele

(1) does not bother me too much since it quickly starts to perform decently (2) not ideal, but not horrible (3) again not ideal, but not horrible

Unfortunately, I think this is as good as we are going to get on iPad iOS 9 for the moment. @jbphet spent quite a bit of time trying to optimize, and we are just going to need to call it at this point. I agree then in a few of these situations the performance is sub-optimal, but I don't see anything damaging.

Much like we did with GAO, we have to accept a compromise here between moving forward and getting the sim published, and spending potentially large amounts of further time trying to squeeze out optimizations for a single platform.

Also good to know that on platforms after the iPad2 we are in good shape. Since the iPad 2 only supports up to iOS 9, that platform will begin to be phased out.

Closing