transform TRAPI output to input for d3 sankey diagram

[andrewsu]

I'd like to create a d3 sankey diagram from TRAPI output. The example input file for d3 sankey (also pasted below) seems like it can be easily generated (in principle) from TRAPI.

Example input file for d3 sankey

``` { "nodes": [ { "node": 0, "name": "node0" }, { "node": 1, "name": "node1" }, { "node": 2, "name": "node2" }, { "node": 3, "name": "node3" }, { "node": 4, "name": "node4" } ], "links": [ { "source": 0, "target": 2, "value": 2 }, { "source": 1, "target": 2, "value": 2 }, { "source": 1, "target": 3, "value": 2 }, { "source": 0, "target": 4, "value": 2 }, { "source": 2, "target": 3, "value": 2 }, { "source": 2, "target": 4, "value": 2 }, { "source": 3, "target": 4, "value": 4 } ] } ```

The fields as I see them:

• nodes.node: arbitrarily assigned numeric identifier
• nodes.name: concatenate the key from TRAPI message.knowledge_graph.nodes with the human-readable name (e.g., CHEMBL.COMPOUND:CHEMBL1200662 / HEMIN from the screenshot example below)
• links.source: the nodes.node entry corresponding to TRAPI message.knowledge_graph.edges.subject
• links.target: the nodes.node entry corresponding to TRAPI message.knowledge_graph.edges.object
• links:value: for now, always assign to 1

Screenshot example


For testing on real-world TRAPI output, here are two examples:

• small example (65Kb file): https://www.dropbox.com/s/b77frv27d078y5s/response.json?dl=0
• large example (24MB file): https://www.dropbox.com/s/104jibw8zm9vfd2/5d72448c-80d7-4cfb-af18-4db2d67bea9a.json?dl=0

Technically, I think this could be implemented either in python or jq.

[rjawesome]

I've made a repository with my code: https://github.com/rjawesome/trapi-d3sankey

Currently I have tested with the small example and I will test with the large example soon.

[andrewsu]

Hmm, well, I thought that the output format here would be plug and play into a d3 sankey template, but as usual, I was wrong. Apparently more tweaking is necessary. @rjawesome if you want to give it a try, here are a few more details...

I'm hoping that we can create an image like shown below. I took a random sankey example from the web (on budgeting, apparently) and overlaid in red how we might use it visualize Translator results (e.g., "What drugs treat NGLY1-deficiency?")

I was hoping that I could simply replace the data file in a template like https://d3-graph-gallery.com/graph/sankey_basic.html, but apparently there needs to be more custom work depending on the data file itself. I'd like to eventually have a visualization that will auto-scale with the size of the input data, but right now even a custom solution would be useful to decide whether this path is work pursing at all.

[andrewsu]

It would also be nice if the input json could be specified as the first (and only) command line parameter, and the output json was written to STDOUT. So we'd invoke it using a command like python3 main.py my_input_file.json > my_output_file.json

[rjawesome]

I changed the syntax of the command. I was able to get a decent visual if the size was changed to 10000 pixels by 5000 pixels and upping the font size

[andrewsu]

cool! Can you post the HTML file you used to render the screenshot above? Either in your repo or in a gist? Very interested to see what the large example file above looks like...

[rjawesome]

I'll probably post it in my repo soon, when I tried the large example file the browser was crashing if I scrolled too fast. Also I had to make the size like 1000000 pixels.

[rjawesome]

Now added to repo!