Show percentages in the python-version, OS graphs

[njsmith]

It'd be handy if the by-python-version and by-OS graphs showed percentages of downloads that fell into each category on each day, instead of absolute numbers. We already see the total downloads in the graph at the top of each page, and then on the graphs below that it's hard to see how the different categories compare to each other because there's so much day-to-day variation in total downloads.

[crflynn]

This particular plot would probably be best. I'll take a look at getting it inserted between the absolutes.

[crflynn]

Currently prototyping this in a branch: https://github.com/crflynn/pypistats.org/tree/proportions

I'll have to either upgrade the db instance or chop the data retention period a bit to make this work.

[njsmith]

...can't you compute them on the fly? you're already loading all the relevant data into memory...

[crflynn]

Sure. I figured storing on db would be faster but I probably shouldn't be too worried about performance here...

[crflynn]

@njsmith changes should be live. e.g. all

[njsmith]

Beautiful!

A few small comments, which I hope are taken as helpful feedback rather than nitpicking :-)

I thought I would like the stacked area plot but... now that I've seen it in action, I think the ideal is actually the original line plot (but still with percentages). In particular, the line plot makes it much easier to compare the different proportions to each other – for example, on numpy, when was the last day that Python 2 had more downloads than Python 3? With the line plot it's not too hard to pick out that it's 8-24, but with the stacked area plot it's pretty much impossible to tell except by mousing over one point at a time and looking at the numbers.

The x-axis for the new and old graphs doesn't match, which doesn't really matter but is a bit odd once I noticed.

I like the way the new graph shows both the percentages and the absolute counts in the tooltips. In fact, given this, I sort of feel like the old graphs can be removed, since the new ones have all the same information and present it in a more useful way :-).

Random other thought that occurred to me: for the plots in general, I wonder if showing a smoothed line would be a bit nicer, e.g. a 7-day moving average (perhaps with the individual points as well).

[crflynn]

I could make the hover text identical between charts; I probably should have just done that here.

Yeah the x-axis not lining up was something I noticed as well. I did a bit of searching and found this to be an open issue with respect to the plotly codebase not being consistent between chart types, unfortunately. Maybe I can do some fiddling and try to line it up.

The reason I kept the old charts is because on smaller projects (like mine) the fill plots look really janky due to proportional variations and are difficult to read.

I'll have to think about the ability to flip between absolute and 7d-avg in the browser. Plotly might support it, otherwise I might need some more javascript knowledge for this :|

I'm planning on attending the pypa October sprint, so hopefully I can chat with some of the maintainers about warehouse, stats, and more features like your suggestions :)

[njsmith]

The reason I kept the old charts is because on smaller projects (like mine) the fill plots look really janky due to proportional variations and are difficult to read.

Do you think it would help to switch away from fill plots, like I suggested?

I'll have to think about the ability to flip between absolute and 7d-avg in the browser. Plotly might support it, otherwise I might need some more javascript knowledge for this :|

Yeah, I know there are a lot of tools like this for js these days but I have like zero idea what they are.

I'm planning on attending the pypa October sprint, so hopefully I can chat with some of the maintainers about warehouse, stats, and more features like your suggestions :)

Oh cool! I won't be able to make it unfortunately, but it'd be awesome to see this integrated more with PyPI.

[crflynn]

Oh, I misread. You meant to discard the fill plots and instead just add proportions to the hover text. That might make more sense. I'll sleep on it for a bit.

[njsmith]

To make sure it's clear: my suggestion is to have one plot, make it a line plot like the old plots, and on that plot we put the percentages, rather than the absolute values. We keep putting both the percentages and the absolute values in the tooltips. That way:

• The percentages are emphasized visually, which seems like what you want 99% of the time

• It's easy to compare the different categories like "python 2", "python 3", against each other because it's a line plot, not an area plot

• If someone does want the absolute values, they can easily get them by checking the tooltips

[crflynn]

I just pushed an update which addresses this. Each segmentation now has a plot for absolute downloads and proportional downloads. https://pypistats.org/packages/numpy

[njsmith]

Beautiful!