search

openaustralia / newsletter

This was a monthly newsletter by the OpenAustralia Foundation for people who love civic tech (Feb 2015-Nov 2015).
5 stars 2 forks source link

Decide what to do next with this newsletter #139

Closed equivalentideas closed 8 years ago

equivalentideas commented 8 years ago

We haven't sent out a newsletter since November and we're trying to work out what to do next. Here's the context and where we're at as far as I understand:

Response

This newsletter was created to support and inform civic hackers and inspire people to create or contribute to civic tech projects. We haven't seen any evidence as to whether or not it is succeeding. We see a few people, who are mostly already involved, tweeting stories from each edition, but beyond this get no feedback. How could we tell if this newsletter was achieving it's aims?

Last year, when we were sending the newsletter, roughly 10-20 people subscribed each month. There are now about 200 subscribers. Over the editions we seem to have settled into a roughly 45% open rate and 15-20% click rate (people who clicked at least one link in the newsletter).

The people who are “engaged” (according to Mailchimp) include regular attendees of our meetup, civic hackers, journalists, hackers more broadly. These people have +70% open rates and click rates.

Costs

In it's current form it is quite resource intensive to produce the newsletter. It takes roughly a day and a half each month to collect links; agree on them; write, edit and refine the summary text for each link; get it into mailchimp; test; and send. For our tiny organisation this is a significant investment.

What next

We observed that we get frustrated working on projects without getting feedback that they are succeeding. This newsletter is one of those projects.

Basically we're reluctant to continue producing the newsletter without knowing if it's creating the impact we want. At our planning session we made a decision about what we should do, but I can't find a record of it, and can't remember what it was.

A few options for a next step come to my mind:

  1. Stop producing the newsletter regularly and use the list for occasional announcements of OAF projects, events or any civic tech related news we think need announcing.
  2. Dramatically reduce the summaries we provide each links and continue to produce the newsletter regularly. The summaries are by far the most costly part to produce. If we can constrain them to be less costly to create and if we can still frame the links successfully with less text, people can still get value from them. We could also cut the summaries entirely but often the links need framing to point out the interesting features.
  3. Try and find out if the newsletter is achieving it's goals. We could do this in a range of lightweight ways.

@henare @kat can you remember any more about what we thought at the planning session? Any more ideas of what we could do?

I was originally feeling like option 1 was the simplest way to go. We've been a bit swamped and stressed under the pressure of running and progressing all our projects since we went from 3 to 2 full time team members. It feels like another thing on the list, and we're not getting the feedback to motivate or validate the considerable effort.

But then I had a chat with someone at the Pub Meet about it. The person was saying that the potential benefit of inspiring a new project or hacker is so huge, but extremely hard to measure. They thought that it was worth the cost for the potential to activate even a little more civic hacking, since individuals acting can make such a huge difference. This reminded me of how I and many others started hacking—through chance meetings, reading, watching etc. . This newsletter is the kind of thing I've found links in that inspired projects or contributions.

That made me think a bit of light research (option 3) for starters could be a good idea to see if we can get some of the feedback we need. I'm pretty reluctant to try Option 2 in our current situation, without any more knowledge about the impact—it just feels like too much to me.

What do you think?

henare commented 8 years ago

@henare @kat can you remember any more about what we thought at the planning session? Any more ideas of what we could do?

@equivalentideas I mentioned to you other day that I remembered us coming up with a plan but you were surprised because that wasn't your recollection :smiley:

My understanding was that instead of our first newsletter of the year we were going to send a final one saying we were shutting it down for the reasons you outline above.

We were also going to invite people to sign up for a new newsletter that was going to be automatically generated from our blog posts (I suggested telling people this list was just going to start doing that so they didn't need to sign up again and offer an unsubscribe but you and Kat were against that).

We would also need to update the subscription links on the blog to be for the new list.

The idea behind this was, as you observe above, lots of the people on the list are actually interested in what we're posting about. Lots of the items did end up being our posts too so in some way it was heading there anyway. Another advantage was there's no extra effort on our part required so it solves the problem of it being a drain on our time. And we have lots of incentive to blog and it gets our blog posts to more people.

So that was my recollection. @kat's probably got another one altogether again :wink:

equivalentideas commented 8 years ago

The idea behind this was, as you observe above, lots of the people on the list are actually interested in what we're posting about. Lots of the items did end up being our posts too so in some way it was heading there anyway. Another advantage was there's no extra effort on our part required so it solves the problem of it being a drain on our time. And we have lots of incentive to blog and it gets our blog posts to more people.

That seems very sensible. Also, 3 of the five most clicked links across all the newsletters were our blog posts.

drzax commented 8 years ago

Here's another data point for your decision making which (I think) supports option two, but doesn't really address many of your concerns.

I'm massively over subscribed to email newsletters generally which is not, I suspect, an unusual problem. As a result, most newsletters get scant attention from me unless they happen to be in my inbox when I happen to be in the mood.

I like having the OAF newsletter appear from time to time and while I won't always engage, there's the ever present chance I will. I think I'm more likely to bite if it's a concise link list with just enough framing/commentary.

equivalentideas commented 8 years ago

Thanks @drzax . Do you think you'd subscribe to get emails when we post a blog at oaf.org.au/blog ?

I like having the OAF newsletter appear from time to time and while I won't always engage, there's the ever present chance I will. I think I'm more likely to bite if it's a concise link list with just enough framing/commentary.

I think we're less concerned about clicks than if it has impacted you in some way to think about/create some civic tech, find a project that's interesting/useful to you, prompted you to make an open source contribution etc. Hard to quantify ;-)

drzax commented 8 years ago

I would probably subscribe to an auto blog email thing (even though I wish I could just use a feed reader—I'm out of the habit).

That certainly is a difficult metric to measure. I don't think I can even answer it for myself.

equivalentideas commented 8 years ago

@henare notes in chat:

Made me think we could do that too - put 3 issues in each newsletter asking for people to work on them

CloCkWeRX commented 8 years ago

A good candidate for content-with-feedback might be conducting the occasional survey; the results of which are intended for advocacy work.

For example: https://www.surveymonkey.com/results/SM-CYY5HQFC/ - a few months ago of councils and open data in general (vic based). That in turn helped support retooling http://opencouncildata.org/ into more of a kit form.

After you have a resource like that; next actions become a lot clearer. IE From the findings there and thinking about planningalerts; obvious outcomes would be to encourage readers to engage with their councils, perhaps even CC'ing OAF in the request.

Similar work is done by some of the cycling advocacy groups here in SA, with pretty good success I think.

Two areas of interest off the top of my head:

kat commented 8 years ago

Since we've had no civic tech news go out, or a resolution on this issue, I've changed the sign up wording to 'Occasional News' and the text to Now we're not implying that it's civic tech based, and the text from "An email newsletter for people interested in the development of technology to enable civic change." to "Occasional news from the OpenAustralia Foundation to your inbox.". Now anyone signing up has not been promised something we're not planning to deliver. Yay.

equivalentideas commented 8 years ago

Thanks for the thoughtful discussion on this issue everyone, it’s extremely helpful.

We’re decided that we're going to make a new list that gets emails about new posts on the OAF blog, because a lot of the interest was in those posts, we still get a way to communicate with people really directly over email, and it requires very low effort from us to setup and maintain.

The steps to doing this are:

  1. [x] Set up the new list.
  2. [x] Switch the form on OpenAustralia Foundation to point to this list, and any other links we have to it. This includes changing the descriptions to reflect what this will be if necessary.
  3. [x] Set the list to get emails about the newsletter
  4. [x] Notify the existing list that what we're doing and where to sign up for the new list (this could be a blogpost as well).
  5. [x] Post something in the readme here that this project isn't currently running.
equivalentideas commented 8 years ago

All done. Adieu Civic Tech Monthly . Here's the link for the new list http://eepurl.com/ch058H