OpenRA / OpenRAWeb

The former source for the official OpenRA website.
http://www.openra.net
46 stars 34 forks source link

Player statistics #322

Open Mailaender opened 8 years ago

Mailaender commented 8 years ago

https://github.com/OpenRA/OpenRA/pull/10803 and https://github.com/OpenRA/OpenRAMasterServer/pull/35 collect a lot of data that is currently only manually analysed. Displaying the anonymous data in pie charts per release might be interesting to thirdparties as well. https://www.gamingonlinux.com does surveys all the time to assess the Linux gaming market share. Displaying the collected data might also help against the irrational fear "my data is collected" without differentiating what data is collected and that our approach really protects privacy.

kingo55 commented 7 years ago

I like this idea. Are you collecting this in a database somewhere?

If so, you could embed graphs with another Python open source project called Superset: https://github.com/airbnb/superset

Mailaender commented 7 years ago

I like this idea. Are you collecting this in a database somewhere?

Yes, see https://github.com/OpenRA/OpenRAMasterServer but I can't find a link to the live one at the moment. It was http://master.openra.net/db/openra.db last time I checked.

pchote commented 7 years ago

The database was made private after we became aware of a specific security issue related to the data it contained. If we have a real use case for making the data public then somebody will need to write a frontend API that serves the relevant and sanitized data.

kingo55 commented 7 years ago

Thanks - maybe there's a sanitised DB somewhere or a schema.

I'll take a look at the SQLite(?) DB on Github but it sounds like that might be old/empty?

Mailaender commented 7 years ago

I suggest you join http://www.openra.net/community/ especially the IRC channel for coordination.

kingo55 commented 7 years ago

Sorry about the wait, @Mailaender - I've provisioned a server with Superset and have a rough dashboard setup:

screen shot 2017-03-19 at 3 17 08 pm

It's designed to be interactive and easy to use with a wide array of visualisations:

screen shot 2017-03-19 at 3 17 40 pm

There's also got a built in SQL editor integrated into the visualisation workflow, so you can explore the raw data without jumping into the command line, Excel etc.

screen shot 2017-03-19 at 3 13 24 pm

And any charts you build inside it, can be embedded elsewhere on the internet through iframes:

screen shot 2017-03-19 at 3 27 12 pm

At work I've deployed it through Docker, which gives us an easy upgrade/setup path. Requests are handled through Nginx to allow for caching, SSL, SPDY and any serverside redirects.

Anyway, if you want to have a closer look at it let me know and I'll share the login/address. The permissions are very fine-grained, so it's easy to share as much or as little access as you like.

PS. When is a good time to join IRC? It looks like everyone is in EU/US.

Mailaender commented 7 years ago

Looks good. Try to contact @ihptru and @baxxxster in IRC or via e-Mail for deployment. I don't have access to the servers.

kingo55 commented 7 years ago

Thanks for that Matthias. I've tried accessing the IRC channel but with everyone in Europe, no one is online when I'm awake.

On Sun., 19 Mar. 2017, 8:11 pm Matthias Mailänder, notifications@github.com wrote:

Looks good. Try to contact @ihptru https://github.com/ihptru and @baxxxster https://github.com/baxxxster in IRC or via e-Mail for deployment. I don't have access to the servers.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/OpenRA/OpenRAWeb/issues/322#issuecomment-287603728, or mute the thread https://github.com/notifications/unsubscribe-auth/ACQILPb9uOQJYyWPmFoY2cTN6U-8Wf8zks5rnPFTgaJpZM4KR6_C .

Baxxster commented 7 years ago

You can e-mail me at glenn_jensen2@hotmail.com with what kind of setup you'd need, like resources, ports, what kind of access etc and I can see if there's room for another VPS :)

pchote commented 7 years ago

See also https://github.com/mindriot101/openra-analysis, which gives a starting point for data we can export to json and display on http://www.openra.net/players/.

kingo55 commented 7 years ago

That's all quite interesting with the fresh data, Paul.

Sorry this dropped off my radar a bit over the past few months. I've just decided to make a link available from one of my VPSs to demo Open RA data on Superset:

http://lendcrypto.pro/superset/dashboard/1/

It's pretty basic and it may not be as nice as the charts you can view on the ORA site, but I think it makes exploration of the data really easy for anyone. We use it with companies to allow non-technical users to access data and make better decisions. For example, by applying some simple filters, anyone could segment the dashboard by RA games since 2014:

http://lendcrypto.pro/superset/dashboard/1/?preselect_filters=%7B%0A%20%20%228%22%3A%20%7B%0A%20%20%20%20%22__from%22%3A%20%222014-09-21%22%2C%0A%20%20%20%20%22game_mod%22%3A%20%5B%0A%20%20%20%20%20%20%22ra%22%0A%20%20%20%20%5D%2C%0A%20%20%20%20%22map%22%3A%20%5B%5D%0A%20%20%7D%0A%7D

Or developers/users can "explore" a chart in depth and share a stateful URL.

e.g. this chart showing average players per game mod trended over time:

http://lendcrypto.pro/superset/explore/table/9/?form_data=%7B%22datasource%22%3A%229__table%22%2C%22viz_type%22%3A%22line%22%2C%22slice_id%22%3A9%2C%22granularity_sqla%22%3A%22started%22%2C%22time_grain_sqla%22%3A%22day%22%2C%22since%22%3A%22600%20days%20ago%22%2C%22until%22%3A%22now%22%2C%22metrics%22%3A%5B%22avg__players%22%5D%2C%22groupby%22%3A%5B%22game_mod%22%5D%2C%22limit%22%3A50%2C%22timeseries_limit_metric%22%3Anull%2C%22show_brush%22%3Afalse%2C%22show_legend%22%3Atrue%2C%22rich_tooltip%22%3Atrue%2C%22y_axis_zero%22%3Afalse%2C%22y_log_scale%22%3Afalse%2C%22contribution%22%3Afalse%2C%22show_markers%22%3Afalse%2C%22x_axis_showminmax%22%3Atrue%2C%22line_interpolation%22%3A%22linear%22%2C%22x_axis_format%22%3A%22smart_date%22%2C%22y_axis_format%22%3A%22.3s%22%2C%22x_axis_label%22%3A%22%22%2C%22y_axis_label%22%3A%22%22%2C%22rolling_type%22%3A%22mean%22%2C%22rolling_periods%22%3A7%2C%22time_compare%22%3Anull%2C%22num_period_compare%22%3A%22%22%2C%22period_ratio_type%22%3A%22growth%22%2C%22resample_how%22%3Anull%2C%22resample_rule%22%3Anull%2C%22resample_fillmethod%22%3Anull%2C%22where%22%3A%22%22%2C%22having%22%3A%22%22%2C%22filters%22%3A%5B%5D%7D

What do you guys think?

On Mon, 26 Jun 2017 at 03:08 Paul Chote notifications@github.com wrote:

See also https://github.com/mindriot101/openra-analysis, which gives a starting point for data we can export to json and display on http://www.openra.net/players/.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/OpenRA/OpenRAWeb/issues/322#issuecomment-310914823, or mute the thread https://github.com/notifications/unsubscribe-auth/ACQILBmKGjwIjo6nHZiRwr-aCfFXWTnRks5sHpP9gaJpZM4KR6_C .