search

ropensci / auunconf

repository for the Australian rOpenSci unconference 2016!
18 stars 4 forks source link

Package for analysing and visualising Australian election data #20

Open benmarwick opened 8 years ago

benmarwick commented 8 years ago

I just read about this new R package for NZ election data that looks pretty neat: http://ellisp.github.io/blog/2016/04/03/nzelect1/ & https://github.com/ellisp/nzelect/blob/master/README.md

It got me thinking that something similar for Australia could be a good open data project for this unconf. We could go further with some spatial viz and interactivity in a shiny app. Perhaps even some Benford's law analysis ;)

Stephen-Gates commented 8 years ago

State election data via Australia's open data census.

ivanhanigan commented 8 years ago

Great idea, did you know there is a hypothesis in the suicide literature that political regime is correlated with population suicide rates? I am hoping to analysis that relationship myself one day. I'd already sourced some data via the Australian Data Archives: Sharman, Campbell. Australian Election Database - New South Wales House of Assembly Party Election Data. Australian Data Archives. https://www.ada.edu.au/ada/40085 (Retrieved 14 June 2015).

On Sun, Apr 3, 2016 at 8:43 PM, Stephen Gates notifications@github.com wrote:

State election data via Australia's open data census http://australia.census.okfn.org/dataset/election.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/ropensci/auunconf/issues/20#issuecomment-204940517

ZhouFang928 commented 8 years ago

Hi everyone, Good topic. Except for visualization, other possible tasks come into my mind would be

  1. predicting election results and detecting potential influencers,
  2. analyzing the voter participation across different groups of people (gender, age, income, occupation, ...) and forecasting the future trend I come from Singapore and not very familiar with the Australia data source. Appreciate if anyone could give me some guidance on how to get the access to election data from the links listed above. I am also happy to establish the connection and form a team to work together. Thanks Fang
benmarwick commented 8 years ago

@ZhouFang928 we can download Excel or CSV files from sites such as http://results.aec.gov.au/17496/Website/HouseDownloadsMenu-17496-csv.htm (for the most recent federal election house of reps results), and maybe incorporate them into a package, or write a package that can download them. For voter participation, we might need polling data, I'm not sure how much of that is publicly available.

cofiem commented 8 years ago

I'd recommend including something from openaustralia.org.au. They do excellent work making information from Australian govt Hansard more useful, and have an api and open source.

dicook commented 8 years ago

there is code and data from 2014 available here http://dicook.github.io/stat585/slides/7-data-structures.html#/8, would need to get updated from http://aec.gov.au/Electorates/maps.htm

dicook commented 8 years ago

To look at polling data or results for each electorate it would be good to build a cartogram. Some electorates are very small and would effectively disappear from a regular map

benmarwick commented 8 years ago

Wow, @dicook those slides are great, looks like you've already done a lot of the work for this kind of package!

Here are some more data sources that we might need for this:

Census data:

Election data:

tslumley commented 8 years ago

A cartogram-like approach from the Grauniad: http://www.theguardian.com/world/datablog/interactive/2013/sep/06/australian-election-results-map

dicook commented 8 years ago

working on it - not too successful yet but hopeful

On Apr 21, 2016, at 2:28 PM, Thomas Lumley notifications@github.com wrote:

A cartogram-like approach from the Grauniad: http://www.theguardian.com/world/datablog/interactive/2013/sep/06/australian-election-results-map

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

Di Cook visnut@gmail.com

dicook commented 8 years ago

ok, take a look at this:

sydney-dorling

i had to split the cities out and work separately on them - darling algorithm explodes with the australian population spread

tslumley commented 8 years ago

image

Some sort of outline is probably needed, but it will have to be done by hand to account for the distortions.

tslumley commented 8 years ago

I've submitted a pull request with the hexmap data and an example.

benmarwick commented 8 years ago

Some examples of the election data viz. Code for these in https://github.com/ropenscilabs/ausElectR/blob/master/AECdata/tidying_and_testing_AEC_data.R

Proportion of total formal votes for the major parties by electorate (just a sample shown here) rplot2

Comparing party and candidate votes of several parties (based on @ellisp's work) rplot3

Voting by electorate and state (based on @ellisp's work) rplot1

benmarwick commented 8 years ago

I've spent a few moments changing things in the eechidna pacakge to satisfy devtools::check and I'm pleased to report the current result:

clipboard01

I've also made a start on the ropensci checklist and about 2/3 of the way through that. I've contacted the ropensci guys about getting the package into its own repo.

I'd like to propose replacing nat_data with nat_data_cart, because it seems to be the same thing, but with a few more columns. But I haven't been able to find any code that shows how nat_data_cartwas created, it would be good to have a record of that. I can see in the Getting Oz Electorate shapefiles vignette how nat_data was made. Who can update the repo with the code that made nat_data_cart?

I've also added two party preferred election data at the polling place and electorate levels as data objects in the package.

dicook commented 8 years ago

I can handle the nat_data_cart. Keep both data. Need to put dorling code into our package for now. I can do this.

From the little black box...

On 25 Apr 2016, at 4:10 AM, Ben Marwick notifications@github.com wrote:

I've spent a few moments changing things to satisfy devtools::check and I'm pleased to report the current result:

I've also made a start on the ropensci checklist and about 2/3 of the way through that. I've contacted the ropensci guys about getting the package into its own repo.

I'd like to propose replacing nat_data with nat_data_cart, because it seems to be the same thing, but with a few more columns. But I haven't been able to find any code that shows how nat_data_cartwas created, it would be good to have a record of that. I can see in the Getting Oz Electorate shapefiles vignette how nat_data was made. Who can update the repo with the code that made nat_data_cart?

I've also added two party preferred election data at the polling place and electorate levels as data objects in the package.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

benmarwick commented 8 years ago

Great, thanks!

Regarding the dorling code, I already copied over the relevant functions from From https://github.com/chxy/cartogram/ into our cartogram.R.

I know this is a bit dodgy, but I did this because I couldn't get a depends on a github package to work (looking at these instructions).

I think it would be better if we could depend on her github package in case those functions change in the future, any ideas how to do that and still pass the checks?

dicook commented 8 years ago

I dont think it is possible to depend on github package. I will work on it later today. On my way back from spotting koalas on raymond island

From the little black box...

On 25 Apr 2016, at 10:16 AM, Ben Marwick notifications@github.com wrote:

Great, thanks!

Regarding the dorling code, I already copied over the relevant functions from From https://github.com/chxy/cartogram/ into our cartogram.R.

I did this because I couldn't get a depends on a github package to work (looking at these instructions).

I think it would be better if we could depend on her github package in case those functions change in the future, any ideas how to do that and still pass the checks?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

dicook commented 8 years ago

Ben,

Just looking at the code. It looks great! Thank you for all the work that you have done to clean it up!

The code to make the nat_data_cart is in the cartogram,R example code for aec_carto_join_f(). I think you might have worked this out already? Looks like it. It doesn’t write the data out but makes a plots of it.

Couple of things/requests:

(1) Is it possible to make the "Plotting-electorates.Rmd” lower case “p”? I tried to remove/add this last week but it still comes up as uppercase for me. I don’t know how to force the change.

(2) This is a request for Carson: is it possible to get the dorling cartogram version of the electorates with a map background into app as an alternative to the hexagon - to be able to choose either on startup?

(3) I have asked Xiaoyue if she minds us including the cartogram code in our package. The way you have done it is good, its nice ot ask too.

(4) It would be really good to have the polygon and data extraction functions from the electorate spatial data as functions in the package. They are in the vignette right now, but it would be good to have functions also. This might be a request for Heike.

(5) I spell-checked the vignettes.

cheers, Di

On Apr 25, 2016, at 10:16 AM, Ben Marwick notifications@github.com wrote:

Great, thanks!

Regarding the dorling code, I already copied over the relevant functions from From https://github.com/chxy/cartogram/ into our cartogram.R.

I did this because I couldn't get a depends on a github package to work (looking at these instructions).

I think it would be better if we could depend on her github package in case those functions change in the future, any ideas how to do that and still pass the checks?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

Di Cook visnut@gmail.com

benmarwick commented 8 years ago

Thanks Di! I was also stymied with renaming that vignette, but then I saw that files can be renamed by editing them in github, and that seems to have worked. Thanks for contacting Xiaoyue, I agree that's the right thing to do.

Good point about the polygon and data extraction functions, it would be great it @heike has the time to move them from the vignette to R/. The internet service here is not good enough for me to download the package needed to run Heike's vignette, so I can't really help with working on that until I get to a better connection in a few weeks.

benmarwick commented 8 years ago

Hi everyone,

I've had some correspondence with @sckott (one of the rOpenSci folks) and he made a separate eechidna repository in the ropenscilabs github org for us.

I've now moved our package there, so please git clone https://github.com/ropenscilabs/eechidna and use that repo for working on the package. This should make it easier to prepare for a CRAN submission and the rOpenSci peer review.