maiera / gde-app

Apache License 2.0
22 stars 9 forks source link

what date to use for blog views, and how to update them #176

Closed brucemcpherson closed 9 years ago

brucemcpherson commented 9 years ago

I noticed that time based reporting is being done at a summary level, and I wondered how people were recording ongoing pageviews & +1's for existing activities. For example, I have a single entry for each topic on my site (which may consist of 20 or so individual pages), that has the date of when I first recorded it in the tracker.

Every now and then, I update each single topic's +1 & pageview counts (to date) from analytics to the new values. I don't create a new entry with, lets say, this month's views, since that doing that could overstate the number of activities (it's the same activity.. just additional impact information, and in some cases, additional blog posts in that topic). I don't really want to create a separate item for every blog post, (which is why I use the topic principle), and I certainly don't want to create a new entry each month - there are now about 700 pages with about 1m pageviews a year.

What is everyone else doing, and what should we be doing? Apologies if I missed guidance on this earlier. My expectation is that I'll automate this at some point but I'd like to understand the details before embarking on that.

patt0 commented 9 years ago

Being one of the most prolific members of the group, I think we can learn a lot from your use case. Your primary mechanism apart from conferences and meetups is your blog which has gets numerous new post every month.

Typically we would see a one to one blog to activity mapping using the tracker, with the metrics related to the activity being updated for about a year. In your case we are looking at topics.

The intent of the tracking app is to be able to quantify impact of GDE's both individually and in aggregate for a particular interval ( quarter, year, ytd, rolling 12 months, year on year ). In your case if we stuck to your categories we would see them disappear from the impact metric after the comparative period is over, which would not reflect the actual (new) impact that you had during the quarter.

When it comes to updating the activity meta_impact as it is now called, having but a few blog activities, I go to my metrics page and update these activities on a ad hoc basis. I have not taken the time to do it even after we pushed the new total_impact formula and I guess its a case that its not that important enough in the face of my other responsibilities.

This is why we want to get an architecture / platform / app that enables for automatic updates of these metrics, which being a very diverse group with many ways and platform through which we engage and impact with our community is going to prove a herculean task.

As i continue to think about it, I wonder wether a GAS spreadsheet that each GDE could copy and use towards downloading, updating manually with their metrics and then uploading back to the tracker would not be a more universal solution. Each GDE could then further script out / pull data from the various outlets they use into their personal spreadsheet ( with an opportunity to expand a linked GAS library with these scripts for the benefit of each ).

I realise that I have not even talked about "monthly" activity reporting for such things as Stack Overflow, Tweeting, Facebooking and what not ... and I have probably muddled the issue many fold.

As you will see in this document I am proposing to get some direction from Program Management on what impact means for 2015 before we decide how best to measure it.

https://docs.google.com/document/d/1SW8cZetMZBH2bZSOiqHMpLz_ieElHF3pGCvjsRxr-no/

Patrick Martinent

On Wed, Dec 10, 2014 at 2:34 PM, bruce mcpherson notifications@github.com wrote:

I noticed that time based reporting is being done at a summary level, and I wondered how people were recording ongoing pageviews & +1's for existing activities. For example, I have a single entry for each topic on my site (which may consist of 20 or so individual pages), that has the date of when I first recorded it in the tracker.

Every now and then, I update each single topic's +1 & pageview counts (to date) from analytics to the new values. I don't create a new entry with, lets say, this month's views, since that doing that could overstate the number of activities (it's the same activity.. just additional impact information, and in some cases, additional blog posts in that topic). I don't really want to create a separate item for every blog post, (which is why I use the topic principle), and I certainly don't want to create a new entry each month - there are now about 700 pages with about 1m pageviews a year.

What is everyone else doing, and what should we be doing? Apologies if I missed guidance on this earlier. My expectation is that I'll automate this at some point but I'd like to understand the details before embarking on that.

— Reply to this email directly or view it on GitHub https://github.com/maiera/gde-app/issues/176.

brucemcpherson commented 9 years ago

Thanks Patrick. A spreadsheet that we figure out how to automate the updating of would be just fine for me, but as first - some clarity on standardization on how to datestamp and measure activities that go on over a period of time would be useful.

keep up the good work

brucemcpherson commented 9 years ago

Patrick

I've been thinking about this a bit and I think you are on the wrong track with a logarithmic approach. The implication is that the 100th +1, the 10000th pageview etc is not as impactful as the first. I would argue the the opposite is the case. You are more likely to read a post more thoroughly if its been endorsed by 500 people than none - so in fact impact increases based on previous impact - it doesn't tail off. I recognize the desire to dampen the effect of a special event like some publicity, but that doesn't actually decrease the impact - and if that's what we are really measuring then it should anyway be reflected.

How about a different approach based on the impact value of an activity -- for example each would have a weight that represents both the work in making it happen and the likely impact on the consumer of the material.. i don't know what the right numbers would be, but for example it may be worth throwing something like this in the pot and see what happens..

a blogpost pageview = .001 points a book copy sold = 1 point a techtalk = .05 point per attendee attendance at a launchpad = 5 points a +1 = .2 points a reshare = .5 points a comment = .3 points an activity = 1 points

etc...

As far as monthly stuff is concerned, that could be achieved but just taking a snapshot of each of the measures at some date, then reporting the the difference each month. An activity would only get scored the first time it appeared, but subsequent impact changes could be logged each month - this would deal with the ongoing +1s, views etc...

patt0 commented 9 years ago

Two separate issues are at play here from what I can see apart from the fact that my formula is probably wrong.

A - Logs are not very well understood http://betterexplained.com/articles/using-logs-in-the-real-world/ http://smotko.si/google-analytics-should-use-log-scale/ http://datadrivenjournalism.net/resources/when_should_i_use_logarithmic_scales_in_my_charts_and_graphs http://stats.stackexchange.com/questions/27951/when-are-log-scales-appropriate

As far as I can see using a log scale to visualise data, has a bearing on the value we attribute the incremental points wether its the first point or the last. We use log scales to be able to interpret the data set as a whole.

B - How we attribute points for each activity is the second part of your post and yes I do agree that their are a number of way's we can score. During the discussion with Ola we understood that different product teams may have different objectives and that each may require a scoring template of their own.

We will look at a mechanism to create these scoring templates so that each team / stakeholder can apply it and get the utility they need.

We may still end up with a solution where we visualise the activity based scoring plotted on a log scale because of the insight in provide on the data set.

brucemcpherson commented 9 years ago

Patrick thanks for the links. Logs are certainly useful, but I would say that they are at their best when used to show growth over time ... so when something is starting off and it doubles each month, you want to be able to see that it doubled, which you wouldn't see on a geometric scale as the early months would soon be dwarfed. So if you are measuring (exponential) change, or even trying to normalize two differently scaled datasets to compare growth - I completely agree with you, but if you are measuring straight relativity it's a little harder to argue.

So if we were to apply log scale to democracy, then the first person to the polling booth would count more than the last .. (you could almost argue that the impact of last one would count more than the first) .. but to show growth (is it slowing down/speeding up?) caused by viral activity you probably would want that logged. In any case, just my 2c worth.

On 15 December 2014 at 09:57, Patrick Martinent notifications@github.com wrote:

Two separate issues are at play here from what I can see apart from the fact that my formula is probably wrong.

  1. Logs are not very well understood http://betterexplained.com/articles/using-logs-in-the-real-world/ http://smotko.si/google-analytics-should-use-log-scale/ http://datadrivenjournalism.net/resources/when_should_i_use_logarithmic_scales_in_my_charts_and_graphs http://stats.stackexchange.com/questions/27951/when-are-log-scales-appropriate

As far as I can see using a log scale to visualise data, has a bearing on the value we attribute the incremental points wether its the first point or the last. We use log scales to be able to interpret the data set as a whole.

  1. How we attribute points for each activity is the second part of your post and yes I do agree that their are a number of way's we can score. During the discussion with Ola we understood that different product teams may have different objectives and that each may require a scoring template of their own.

We will look at a mechanism to create these scoring templates so that each team / stakeholder can apply it and get the utility they need.

We may still end up with a solution where we visualise the activity based scoring plotted on a log scale because of the insight in provide on the data set.

— Reply to this email directly or view it on GitHub https://github.com/maiera/gde-app/issues/176#issuecomment-66971427.

SmokyBob commented 9 years ago

Moved to new repo