langcog / oecs-viz

https://langcog.github.io/oecs-viz/
0 stars 0 forks source link

using your networkD3 approach with a graph database? #1

Open Analect opened 2 months ago

Analect commented 2 months ago

@mikabr ... I happened up on this repo, since I had been researching some ojs and quarto stuff here.

I know ojs isn't used so much in here, but I'm interested on your implementation of network graphs here and the ability to link to content from them. I can see the graph is setup in the root index.qmd file, which in turn references functions within the _processing/oecs-viz.qmd folder. While I'm no expert on r, it seems you are calculating centrality relationships between the topic documents?

I'm probably after a more simple fix. I have been playing around with quarto-live and the embedded graph database kuzu over here. Kuzu is akin to Neo4j, but is embedded, like DuckDB and so doesn't need a running server. There isn't a fully functional webassembly version yet. If you run some of the cells, you can see I have something basic functioning with the python package networkx, but I find it is somewhat limiting for displaying a graph, especially one that can link off to other content.

From your experience, is it straight-forward to wire up networkD3 to output from a cypher query against kuzu (akin to running a cypher query against neo4j). I'm not sure if you have dabbled in these. I'd be interested in your thoughts. Tks.

mikabr commented 2 months ago

@Analect I'm not familiar neo4j or kuzu so I can't speak to those issues specifically. In this project the data changes pretty slowly, so we just fetch and cache it offline.

There's a lot of extraneous stuff in _processing/oecs-viz.qmd from playing around with different graphing techniques. What we ended up using is GLoVe vectors, specifically pairwise cosine similarity between documents' average vectors. We're then using functions from the package tidygraph to get network groups and centralities. The actual plotting uses forceNetwork() from the package networkD3.

Hope that helps!

Analect commented 2 months ago

Thanks @mikabr ... I'll work my way through your links. Appreciated the input.