Database/MongoDB support

[Phrancis]

I've looked at several SQL and NoSQL environments for data stores. SQL seems too rigid for the kind of data generated by Cardshifter servers. Some NoSQL DBs seem more fitting. Neo4j graph database might be neat to implement at some point in the future (once we have a lot of data), but I think that for right now MongoDB's BSON/JSON data store would be best acclimated to the kind of data we generate by playing the game and such. It seems like it would fit pretty nicely with the way we have been handling data.

I want to take the pulse on how we could do this. JSON seems like it would fit very nicely for these data sources which we currently support:

• Replays
• Card sets / card libraries

With some work I think it could also support things like users/players, cards (with versioning?), maybe even bigger things like tournaments and leaderboards. I think it would mostly be a matter of defining a way for the server to pass data in a way that's useful to non-computer entities, so the data from the ECS system makes sense and can be aggregated in useful ways.

For example, while the game server closely keeps track of all entities present (with IDs), those are difficult to associate with the actual entities (in human terms) from the game.

I would suggest perhaps we could try a trial implementation (using the MongoDB API for simplicity) so we can see and tweak how the data goes in. MongoDB has a BasicDBObjectBuilder that we can probably just create when a game starts, write a few simple classes to receive data (both raw entity data, and associated "human-friendly" data to go with it), and then build it when the game is over and insert it in.

If and when we're happy with what we are getting, then perhaps we could implement Hibernate OGM for a more "permanent" setup.

Thoughts?

[jacwah]

I think the first question we should ask is not what to use, but what we want to achieve. You've named the main things:

• Persist decks
• Save replays
• Possibly leaderboards/player stats

Using MongoDB seems like a good option. I don't know how performant it's for the equivalent to select (in SQL)

[Phrancis]

From the little bit I have used, MongoDB seems to be quite fast, since the data is indexed similarly to how it would be in a SQL DB.

I made an example script which shows 4 different ways to build and insert the data. They all seem equally as fast AFAICT, but 2. Using BasicDBObjectBuilder might be the most sensible to use.

In the bigger scheme of our code base, where would this best fit? I'd like to branch and try to write some code and see how far I can get, just not really sure "where" to start

[Phrancis]

Here is a useful reference guide for MongoDB CRUD Operations: http://docs.mongodb.org/manual/core/crud-introduction/

MongoDB has its own query language that can be used natively in Groovy / Java, for example for retrieving a user's decks from a decks collection when starting a game we could do something like:

def deckOwner = CardshifterDB.users.find( { user_name: "Phrancis" }, { _id: 1 } ) // get user's ID
def userDecks = CardshifterDB.decks.find( 
    { owner_user: deckOwner, mod: "Mythos" },  // search criteria
    { deck_name: 1, date_created: 1, deck_version: 1, deck_cards: 1, _id: 0 }  // values we want, excluding the document's ID
).sort( { date_created: -1 } ) // sort by date in reverse order, i.e. newest first