Meta Graphs from FactMiners

[moxious]

@FactMiners on twitter provided this link:

http://gist.neo4j.org/?8640853

It's some documentation / a convention for "meta graphs". Something akin to a Neo4J schema. Make sure to explore what the conventions here are, and/or look into reusing that cypher.

[Jim-Salmons]

Hi moxious,

@FactMiners will eventually (hopefully, surely, with luck and persistence) be a "serious fun" dev community, but at the moment it is me (@Jim_Salmons). I appreciate that you resonate with the ideas we're working on at FactMiners -- the embedded metamodel subgraph seems like such a useful idiom/pattern whatever we call it -- but a quick word of caution. The idea of a META partition as described in my GraphGists is a non-mainstream edge-case.

What you are working on with your NeoProfiler has broader general use, so attend to that "itch" before worrying too much about what is suggested by the embedded META subgraph idea. For example, I think a bit of GUI wrapper on this and it would be a great addition to a Neo4j learner's "toolkit" after installing Neo4j and loading/unloading various datasets while doing tutorials, etc.

In fact, while it is little used (except by me as it scratched my itch), I want to look at what it would take to put a "Profile" button on my super lightweight Windows-based Neo4jCP Control Panel (https://github.com/Jim-Salmons/neo4jcp). I did this AutoHotKey script when first starting to explore Neo4j. It currently only works for the non-installer version that let's you set up Neo4j as a Windows service, which I prefer, because I can more easily find and control it for the kinds of things done for database quick-change, creation, deletion, etc.

Would be cool to do a Neo4jCP database change and get a profile view as a glimpse of what you just brought on-line. Seems I would just need to detect if NeoProfiler was available, tweak the Neo4CP UI to expose the feature, then ask for and display the NeoProfiler report. That sounds like a reasonable near-term to-do. Will keep you posted! :-)

Congrats again on NeoProfiler. The Neo4j community thrives on active learners -- you are an active learner, too, right? If not, sorry to not know. --Jim--

[moxious]

In general, I agree that a "metagraph" would be a good thing in most/all neo4j databases. In the RDBMS world, we take for granted that all of our metadata is in queryable tables. So your entities/relations are themselves tables. So your idea of putting a meta graph as a graph into a graph database is something I'd consider just following good practice. You've got some data that's useful and needs storage -- where else to put it than in the storage engine you're already using?? :)

If I understand correctly, you'd prefer that these things be modeled and stated explicitly ahead of time. This would be more akin to how people build and assert models; a good idea. Yes, the neoprofiler thing is a bit different in that I'm starting from the assumption that people don't have it. Convention and practice with neo these days doesn't even really encourage people to think about modeling except in a general way. In fact one of the things I think the graph community really lacks is a body of knowledge about good modeling practice. The RDBMS world has that, but they again they have a 30 year head start.

Neoprofiler scratches my itch and tries to derive this model by sampling an unknown database. It might also be useful as a transitional sort of thing. Let's say a user hasn't created a real graph metamodel (as I think we'd both prefer they would). They could use neoprofiler to infer a starting point, and then go from there. I'm working on some code (you saw a screenshot on twitter) that creates a graphical depiction of a metagraph of sorts.

My main use case is just knowing what the heck is in a database, but of course models have much more general benefits, like enforcing integrity. Eventually, if you know that all :Person nodes connect via :KNOWS relationships, (and not other relationships) then you can query for relationships that should not be there, and trace down programmatic sources of bad data quality.

So in general I think the metagraph idea is a good one. When I get some more time to work on this code base, I'll go through the 4-part series. My tool won't change the way people model graphs, but it would be nice if the result of my tool looked similar/the same as the metagraphs you're describing.

[Jim-Salmons]

Hey moxious... Yes, the "deck chairs" are considerably moved around when jumping from RDBMS to a full graph database like Neo4j. More of the things you are used to dealing with are handled by incredible framework/platforms like www.Structr.org where explicit domain modeling, schema constraints, etc. are handled. I plan to build FactMiners on Structr/Neo4j. This collaboration will result in aspects of FactMiners tools/components finding their way, via generalization, into Structr for non-gaming use.

At 63-years old and after a 3-year successful cancer battle, I'm all about "serious fun." FactMiners (and its first-of-a-kind to-be Fact Cloud at www.SoftalkApple.com) is a way to continue my "deep weeds" technical interests without Corporate Road Warrior distractions and pressures to compromise quality for deadlines or funding (we have neither!?). :-)