API: Use profile tags to create a visual representation of our community with GraphCommons

[ebarry]

Please describe the idea

Figure out how to align our API for use with GraphCommons.

What problem could this idea solve?

Creating an at-a-glance visual representation of people in the PL community and what roles/projects/places/methods they are associated with will help with onboarding new people into collaborating in the Public Lab community.

We don't have to create the visualization from scratch, rather, thanks to awesome work by many people including @mridulnagpal we already have profile tagging, and thanks to awesome work by other folks there are platforms like GraphCommons who can take in our profile tags via an API to create a community visualization.

Because making this happen will require our API, i'm putting this in the API milestone

To make it easier to understand Public Lab at a glance, i suggest we make use of these profile tags via our API in order to use something like https://graphcommons.com/ to create a visual representation of our community.

Here's another example visualizion that @steviepubliclab posted in https://github.com/publiclab/plots2/issues/104

[jywarren]

Just a note that after some research, GraphCommons uses CSVs for import:

https://github.com/graphcommons/graphcommons-issues/issues/31#issuecomment-210790489 Edges list structure:

FromType, FromName, EdgeType, ToType, ToName
"Event", "E1","attended", "Person", "EVELYN"
"Event", "E1","attended", "Person", "LAURA"
"Event", "E7","attended", "Person", "SYLVIA"
...

Nodes list structure:

Type, Name, Description, Image, Reference, Date of Birth
"Person", "EVELYN","","","","1930-6-27"
"Person", "LAURA","","","","1930-6-27"
"Person", "THERESA","","","","1930-6-27"
...

So, would profiles look like the "nodes" above, and would edges be commonly held profile tags?

[jywarren]

And let's start with the simplest graph type possible -- would that be like, people related through profile tags? Or something else?

[jywarren]

Oops, closed by mistake!

[david-days]

Hey all--jumping in here if you don't mind. I've done a lot of digging into graph display libraries and options over the years, so I thought I might help.

The edges for the charts, such as the example, could be any defined relationship. Tags-in-common would be one that could be drawn directly from the profiles. Another would be to dig into projects, questions, or just about anything that has interaction between individuals. You can start with a basic graph easily enough (simple relationships), and then work up from there.

The example graph is basically a tree graph with additional edges between leaf nodes. So if you start by working on the construction of the tree graph display, then you can add additional edges based on whatever relationship criteria you come up with.

I noticed that graphcommons has a price tag associated with it. One open-source option is D3.js, which is pretty flexible and widely used. The examples show a wide variety of what you can do with node/edge data--the fun part for any library is figuring out the data relationships and kicking off the display process.

Whichever library you choose, though, it would be easy enough to add a javascript to turn API data into the necessary format, I believe.

Edit to Add:

For more complex (less linear) relationships, I've always found dependency wheels to be interesting. Here are a few examples:

Straight up Relationships Hovering over a section will highlight the to-from relationships

Rotating Wheel You can use the mouse to move the wheel around.

[jywarren]

Cool, thanks @david-days ! I'll defer to Liz about whether to use GraphCommons or another library. On our end, it seems like there are two immediate next steps possible;

1. [ ] make sure we generate CSV output of nodes (people) and edges (relationships between people, starting with commonly held profile tags) in our API (a good project for http://publiclab.org/tag/soc-2018-api
2. [ ] explore additional automated scripts (independent of our site/codebase) to fetch the above CSVs and upload them to GraphCommons in line with Liz's request above
3. [ ] just to add this in here, we could use @sagarpreet-chadha 's demo code here once we have edges and nodes to generate this ourselves as well: https://github.com/publiclab/plots2/issues/1502#issuecomment-360087043, screenshot here:

In both 2 and 3, we'd have to think through how to manage generating this regularly and the database load it might create, as discussed in #1502.

Thanks, all!

[sagarpreet-chadha]

Hi everyone !

So i have generated this code using vis.js library . The code for above is : https://gist.github.com/sagarpreet-chadha/d70d08b627e2eea220cfa79aee1b858c .

and also i attempted to create an API for nodes and edges here : https://sagarpreet-chadha.github.io/nodesedges.json

This is how it looks :

In case of any doubt , feel free to ask me :) This is a pretty cool feature and i would love to help anyone who may want to work on it 😄 . Thanks !

[ebarry]

Wow this is really cool. It's amazing that we could have our own open source graph visualizer like this. How would we use this on /people/network or at the top of /tags? Can it be embedded?

[sagarpreet-chadha]

Thanks @ebarry !

Yes it can be embedded anywhere on our website 😄 .

[ebarry]

How many nodes (tags) have you tested it with? Like, the top 150 or 500, or perhaps all of them?

[sagarpreet-chadha]

The above graph is just a demo graph with a demo API for testing purposes containing about ~16 nodes . But we can use any number of nodes - see this example on their site here : ~500 nodes and it is not lagging . http://visjs.org/examples/network/data/importingFromGephi.html

[sukhbir-singh]

Hi all, I have added community visualisations as one part of my gsoc proposal and would like to work on it during summer as a project. It would be very helpful if you people can review my proposal and can provide your valuable feedbacks on it.

My proposal link: https://publiclab.org/notes/sukhbir/03-02-2018/gsoc-proposal-v2-api-development-and-providing-support-to-3rd-party-app-integrations Thanks!!