search

DOI-USGS / water-use-15

USGS water use data visualization emphasizing the newly added 2015 dataset.
Creative Commons Zero v1.0 Universal
12 stars 15 forks source link

Improve pie chart / circle performance #97

Closed aappling-usgs closed 6 years ago

aappling-usgs commented 6 years ago

Make it faster to load the page, and transition into and out of pie charts and circles,

Relates to #71 (basic pie charts), #72 (spiffy pie chart transitions), and #62 (load time performance), and messing with transitions and loadable data are within scope (despite overlapping with those other issues) to the extent that they affect performance.

aappling-usgs commented 6 years ago

Lindsay and Alison have already been working on this for several hours today (https://github.com/lindsaycarr/water-use-15/commit/eafe0893aa0c10940ded20e90ad1dc54bfedf489 through https://github.com/lindsaycarr/water-use-15/commit/0269bf314869fabf7e1d3a07213ea4adf8b1c188). We've made it so the pie groups and pie slices only get created once per session, moved the pie chart data preparation into build_map so that also only happens once now, and temporarily turned off transitions to be able to view the changes faster. As a baseline from there (I'm hoping to find additional opportunities for improvement), here are

total scripts (all the yellow) rendering (all the purple) painting (all the green) 'load' handler took __ms
1507 1229 77 5.5 873
2745 1351 82 5.3 949
1625 1313 79 5.3 940
1551 1246 79 5.6 873
1556 1254 80 5.3 900
total scripts (yellow) rendering (purple) painting (green) 'load' handler
2967 1749 80 3.4 1201
2166 1782 77 3.2 1372
3204 1776 75 3.3 1390
2096 1728 81 3.9 1340
2137 1776 85 2.5 1404
total scripts (yellow) rendering (purple) painting (green) 'mouseover' handler
?? 473 302 26.6 428
?? 520 310 27.5 464
?? 536 307 28.4 497
?? 517 303 27.5 474
?? 561 301 26.7 534
total scripts (yellow) rendering (purple) painting (green) 'mouseover' handler
?? 227 275 25.7 224
?? 227 293 28.2 225
?? 213 284 26.5 213
?? 217 275 24.7 215
?? 296 272 25.6 195
total scripts (yellow) rendering (purple) painting (green) 'mouseout' handler
?? 167 264 8.2 162
?? 266 337 8.5 145
?? 153 270 8.4 150
?? 167 273 8.0 165
?? 243 273 8.2 152
aappling-usgs commented 6 years ago

After updating so that pie charts are scaled with transform(scale()) rather than having county-specific radii in the arcs (https://github.com/aappling-usgs/water-use-15/commit/9bee09346ce5e50e4c5039438885abaf81a82e62) (and also updating to the new version of vizlab, with text not spanning the full 100%, which I think will affect times very little):

total scripts (all the yellow) rendering (all the purple) painting (all the green) 'load' handler took __ms
1508 1235 82 5.0 888
3015 1282 83 28.7 880
1640 1266 81 5.0 868
total scripts (yellow) rendering (purple) painting (green) 'load' handler
3571 1757 79 4.5 --
2052 1696 77 2.3 --
2032 1698 69 2.3 --
total scripts (yellow) rendering (purple) painting (green) 'mouseover' handler
-- 551 315 26.7 515
-- 529 306 28.8 487
-- 594 307 28.6 564
total scripts (yellow) rendering (purple) painting (green) 'mouseover' handler
-- 488 296 26.9 434
-- 285 296 27.1 282
-- 308 278 27.7 306
total scripts (yellow) rendering (purple) painting (green) 'mouseout' handler
-- 177 279 9.1 174
-- 172 264 9.3 166
-- 195 273 9.0 191
-- 197 265 9.0 193

These numbers look quite similar overall. It's possible that the first transition from Total to Pie view is now taking a little longer than before, but 2 data points from the previous comment have longer scripts times than 2 points from this comment, so the effect is not certain. Subsequent transitions from Total to Pie may also be longer (more like 300 something rather than 200 something ms). Nothing looks substantially faster to me.

aappling-usgs commented 6 years ago

@jread-usgs , laura mentioned in standup that you'd found some nifty way of representing everything as one giant path (or something) in gages through the ages, which was the deal-maker for getting that vizzy to be performant for a lot of gages. do you remember what you did and/or where the relevant issues or PRs might be?

aappling-usgs commented 6 years ago

As of https://github.com/aappling-usgs/water-use-15/commit/9f0c0f5b9168a046aaf4d306c6baef07cf83e4fd I've reverted to county-specific radii, have pushed more of the data preparation work into pieData(), and have attempted to tighten up here and there. Not expecting big improvements, but about to try out some quite different ideas and want a baseline, so here goes:

(my computer also restarted between this session and the previous one. might be contributing to greater speed, or not?)

total scripts (all the yellow) rendering (all the purple) painting (all the green) 'load' handler took __ms
1600 1239 81 27 858+167
1651 1322 80 21 899+188
1589 1240 78 25 838+167
1605 1233 80 10 860
1546 1194 76 36 838

(last two are same http.server session, not sure why i started getting just one load handler instead of two again)

total scripts (yellow) rendering (purple) painting (green) 'load' handler
1930 1334 288 28 996
2219 1386 316 28 983
2095 1430 309 33 999+203
2009 1382 292 28 967+206
2020 1394 289 28 985+201
total scripts (yellow) rendering (purple) painting (green) 'mouseover' handler
-- 454 303 31 373
-- 393 299 26 354
-- 484 304 26 448
-- 428 302 26 353
-- 398 298 26 351
total scripts (yellow) rendering (purple) painting (green) 'mouseover' handler
-- 190 300 31 188
-- 182 275 25 180
-- 263 271 26 172
-- 178 285 27 173
-- 184 273 25 182
total scripts (yellow) rendering (purple) painting (green) 'mouseout' handler
-- 160 269 9 159
-- 138 268 10 135
-- 170 262 10 165
-- 153 265 9 146
-- 156 279 8 150
total scripts (yellow) rendering (purple) painting (green) 'mouseover' handler from what to what
-- 93 36 5 79 from blue to blue!
-- 71 45 7 66 blue to yellow
-- 85 53 9 82 blue to red
-- 95 47 19 81 blue to green
-- 71 46 14 67 blue to grey
total scripts (yellow) rendering (purple) painting (green) 'mouseout' handler from what to what
-- 77 34 5 75 blue to blue
-- 91 48 5 90 yellow to blue
-- 86 52 10 84 red to blue
-- 81 45 7 80 green to blue
-- 84 45 8 82 grey to blue
aappling-usgs commented 6 years ago

ideas still to try:

lindsay

alison

consider doing later

jordansread commented 6 years ago

One potential gotcha on drawing a circle with a path in svg - you can't draw the full circle with the arc.

The simple solution is to draw 99.9% of the circle and then close w/ z ("close" polygon in svg-path-speak)

jordansread commented 6 years ago

and maybe also in order to get rid of all possible slow-down sources, remove the tool-tip function for the time being?

jordansread commented 6 years ago

Can the circles be pre-sorted (like in the geojson)?

aappling-usgs commented 6 years ago

Lindsay suggested that, too, but then we thought more about it and couldn't find a cheap way to make them pre-sorted for all categories. So now Lindsay is trying out a way to not have to sort them at all (by moving tooltip behavior to be county-centric rather than dot centric). If it works technically, this change will involve a check from our design lead :wink: , but Marty and Lindsay and I agreed that it might be sufficiently intuitive and a good move if it improves performance a lot. (Sorting is currently responsible for about half of the Scripting time in each switch among categories, so it probably will.)

aappling-usgs commented 6 years ago

Options for showing/hiding items: 1 .style("display","none"); .style("display", null); 2 .attr("hidden",true); //to hide .attr("hidden",null); //to show 3 .style("opacity",0); //to hide .style("opacity",1); //to show

aappling-usgs commented 6 years ago

jotted notes on css animations & scaling:

    .style("transform", function(d) {
      return "scale3d( " + ... + ")";
    });
    // style attr for scaling: ease-in, ease-out

and from climate fish viz https://owi.usgs.gov/vizlab/climate-change-walleye-bass/wallyTrendsFig-desktop.svg

@keyframes move-wally-7{
  0% {transform: translate(11px, -7px)}
  33% {transform: translate(-14px, 4px)}
  67% {transform: translate(15px, -1px)}
  100% {transform: translate(11px, -7px)}
}
#walleye-7 {animation: move-wally-7 7s ease-in-out infinite;}
jordansread commented 6 years ago

does this work?

.style(function(d) { return {
   transform: "scale3d( " + ... + ")", transition: "all 1s ease-in-out"}; 
}})

Looks like you can specify multiple values at once? from https://github.com/d3/d3/pull/277

aappling-usgs commented 6 years ago

Not working, no. I'm still just working on getting a basic translate and scale to work with style(). I've tried splitting up the translate and scale bits into different calls with a tremendous lack of success, so now I'm back to defining them together, but even with that i can't seem to get them to show up in the inline css. the internet seems to always use .attr("transform", function(d) ...) rather than .style(... for transforms. I can use .style() for other things like fill and stroke-width, no problem, but can't seem to figure out transform.

leaves the inline css empty:

    .style(function(d) {
      return {
        transform: "translate(" + d.coordinates.x + " " + d.coordinates.y + ")"+
                   "scale(0 " + scaleCircles(d.properties[[newCat]]) + ")"
      };
    });

modifies the inline css:

   .attr("transform", function(d) {  
      return "translate(" + d.coordinates.x + " " + d.coordinates.y + ")"+
             "scale(0 " + scaleCircles(d.properties[[newCat]]) + ")";
    });
jordansread commented 6 years ago

I am surprised that the second example modifies the inline css and not the transform svg attribute. I would think you would do this to modify the inline css (style):

.attr("style", function(d) {  
      return "transform: translate(" + d.coordinates.x + " " + d.coordinates.y + ") scale(0," + scaleCircles(d.properties[[newCat]]) + ");";
    });

I think your example does this: <circle ... transform = "translate(1,2)scale(0 4);"/> but I think you want <circle ... style = "transform: translate(1,2) scale(0,4);"/> for the transform to by a CSS transformation instead of an svg transformation.

jordansread commented 6 years ago

Geez, not much luck on my end. I couldn't get the different style things to play nice.

I do have a simple svg here with what I think we are going for in terms of transition from one category to the next:

<?xml version="1.0"?>
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" preserveAspectRatio="xMinYMin meet" viewBox="0 0 200 200">
    <style>

        @keyframes coin-flip {
          0% {
           transform: scale(1,1);
           fill:red;
          }
          49% {
           transform: scale(0.1,1.3);
           fill:red;
          }
          50% {
           transform: scale(0.1,1.3);
           fill: green;
          }
          100% {
            transform: scale(1.6,1.6);
            fill: green;
          }
        }
    #test:hover{ 
      animation: coin-flip 1s;
      animation-timing-function: linear;
    }
    </style>
    <g transform="translate(75,75)">
        <circle r="40" stroke="black" stroke-width="10" fill="red" id='test' vector-effect="non-scaling-stroke"/>
    </g>
</svg>

if you open that in a browser you can mouseover and see. I don't come up with a simple way to do this w/ css that didn't use keyframes. There are chained examples as has been noted, but what I don't see in the chaining in doing a scale, then something, then another scale. hmm...

jordansread commented 6 years ago

I see the issue w/ scale/translate in the same transform element. It does get wiped and needs to be updated to work. instead, you can create an outer group that holds the translate, and put the scale in an inner group. This is working example:

<?xml version="1.0"?>
<html prefix="og: http://ogp.me/ns#" lang="en">
<head>
    <script src="https://d3js.org/d3.v4.min.js" charset="utf-8"></script>
</head>
<body>  
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" preserveAspectRatio="xMinYMin meet" viewBox="0 0 5 5">
    <g transform="translate(2,2)" class='county-centroid'>
        <g class = 'pie'>
            <circle r="1" fill="red" id='test' class='coin tin'/>
            <g class='coin slices'>
                <path d='M 1 0 A 1 1 0 0 1 0.809017 0.587785 L 0 0' fill="coral" />
                <path d='M 0.809017 0.587785 A 1 1 0 1 1 -1.83697e-16 -1 L 0 0' fill='green' />
                <path d='M -1.83697e-16 -1 A 1 1 0 0 1 0.951057 -0.309017 L 0 0' fill='rgb(0, 171, 107)' /> 
            </g>
        </g>
    </g>
</svg>
    <script>
      var aniDur = 800;
        d3.selectAll('.pie')
            .transition()
                .ease(d3.easeLinear)
                .duration(aniDur/2).delay(0)
                .attr('transform', 'scale(0.1,1.3)')
            .transition()
              .duration(aniDur/2)
              .attr('transform', 'scale(1.6,1.6)')

        d3.selectAll('.tin')
            .transition()
                .duration(0).delay(aniDur/2)
                .style('display','none');

        d3.selectAll('.slices')
            .style('display', 'none')
            .transition()
                .duration(0).delay(aniDur/2)
                .style('display', null)

    </script>
</body>
</html>
jordansread commented 6 years ago
lindsayplatt commented 6 years ago

For me, time spent on #102 was 3 hrs.

aappling-usgs commented 6 years ago

I've spent 23 hours on this as of #109

jordansread commented 6 years ago

I have a few experiments (happy to share code to generate this, but I didn't want to check it into the repo). I wanted to isolate the challenge of figuring out what is slowing the transitions down.

Findings:

Visually, they look pretty similar to me unless I really focus on the midpoint of the transition and the path is a bit smoother. If you throttle the CPU w/ chrome devtools, then the circle one starts to get really clunky.

Here is what the two look like before transition: image and after: image

Using a path makes things harder for getting data on the map, but, like w/ gages through the ages, it likely helps w/ performance. Paths also let us keep a consistent stroke (but note this would be possible w/ circles too if we didn't use the coin-flip and instead took the radius to zero and then back up). Probably not enough of a reason to shift though. I will share the example html in our slack channel

update apparently a smooth single transition between colors and geometry works fine too if not using pie charts. same issue w/ for throttled CPU.

aappling-usgs commented 6 years ago

Cool! So: 1) use a single path for all circles (one path per category) 2) use d3, not css after all, to do the transition 3) it was initially looking like it was important to separate the geometry and color transitions, but now you're finding that a single transition on both works well for circle-paths, is that right?

With those conclusions in mind, it sounds to me like color fades while the circles grow or shrink may perform just about as well as coin flips with a quick color transition in the middle. Is that true? And my initial reaction to coin flips (with the caveat that they were choppy in my implementation) is that the color fade and growing/shrinking actually looked a bit nicer. What's your preference now that you've seen both working smoothly?

jordansread commented 6 years ago
  1. I think so, but I want to try one more test that uses the r instead of transform in the transition. I will add it to my experiment and report back. Should be simple, but I need to finish taxes first.
  2. definitely. d3 is smart w/ these things :+1: I'm sold.
  3. yes - w/o the pie charts, this is no longer necessary, and we can take advantage of a single transition for both geoms and style. This is simpler and a little smoother. I agree that the resize and re-color transition looks the best.
aappling-usgs commented 6 years ago

All sounds good! Heh, I just finished my taxes too.

jordansread commented 6 years ago

Ok - changed to scale radius instead of transform. Now the path and circle implementation look identical to my eyes. There is a performance improvement with paths though. The difference is subtle on my computer, but using chrome to throttle the CPU makes the circle transition choppy and a 1s transition takes 1.5s. The path looks fine to me w/ the throttling. path: image circles: image

aappling-usgs commented 6 years ago

So you're saying that r vs transform doesn't matter at all, and that only path vs circles matters? Or would r vs transform matter in some cases?

jordansread commented 6 years ago

@aappling-usgs r vs transform doesn't seem to matter for performance, although I would guess there is a slight advantage to r since D3's parser and interpolater has to do slightly less work. The main difference between the two here is that r keeps a consistent stroke width (preferred) vs transform='scale(x,y)' scales the stroke, which is less than ideal. That is what I meant by "Now the path and circle implementation look identical to my eyes", but I should have been clear about that. The path implementation also keeps the stroke width the same, so that is what I was referring to.

aappling-usgs commented 6 years ago

ahhh, cool, thanks. great, r and path seems like a winning team

wdwatkins commented 6 years ago

Haven't read this whole thread, so sorry if this is already noted, but I noticed some things while messing with the analytics:

Looks like we reload a category whenever the button is hovered over, even if that category is already loaded. Same with the mouseout. There should be a check there to see if the category is already loaded.

And there seems to be mouse-in/mouse-out happening when you click on a category, not sure that should be happening. I would think it is less snappy since it is loading the new circles twice.

wdwatkins commented 6 years ago

Might we also want to add a very small delay to deciding to load a new category, so just incidentally dragging the cursor over the buttons doesn't trigger category changes? Although if the category changes get snappier this might not be an issue.

lindsayplatt commented 6 years ago

@wdwatkins that's a good idea. I think we should create a separate issue for it (@aappling-usgs do you agree?). Here's an example for click vs double click and I think that's pretty much what we would use for mouseover vs click. We should also handle mouseout differently if it's been clicked. No need to revert the category.

https://github.com/almende/vis/issues/203#issuecomment-48481501

aappling-usgs commented 6 years ago

OK, issue created (#126). Thanks @wdwatkins and @lindsaycarr - great observations.

aappling-usgs commented 6 years ago

two major ideas have been spun off into their own issues:

what i think remains:

...however, neither of those experiments has a clear action item we could attach to the outcome. we need tooltips (or .on('hover') events, anyway) and circle transparency for the vizzy to be usable. so i'm going to close this issue. if anybody feels strongly that i've missed something or that the above experiments are worthwhile, please reopen and make your case.