Avoid failures in copying graphs

[mr-niels-christensen]

The cron job that copies "newdata" to "default" failed when it took more than 2 minutes to complete. The problem seems to be that iterators (cursors) over NDB queries time out after 2 minutes (because data may have changed anyway).

The current best solution suggestion is to not copy the data at all. All access (server- and client-side) must use a meta-graph to determine which graph to use. Instead of "default" and "newdata", graphs would be "A" and "B".

Possible variation: Could ndbstore provide enough metadata that the meta-graph would be unnecessary? Possible variation: Would it be better to use disposable graphs, replace "A" and "B" with a datetime or a hash?

Possible alternative solution: Optimize the copy. Do not read and write triples, just copy the N3 blobs in the db.Model objects. [But only experiments will tell how much this scales] Possible alternative solution: Shard the copy operation into smaller pieces. [But this does not scale well]

Not a solution: Use direct NDB->NDB copy. [There isn't a copy operation].

[mr-niels-christensen]

Actually, as long as you stick to a fixed list of graphs (like "A" and "B"), the metadata could live inside the graphs, keeping the overall setup simpler.

[mr-niels-christensen]

Suggested solution:

• Keep "metrics" as is.
• Keep "default" and "newdata" as graph links, not graph IDs
• "default" will always refer to either graph "A" or graph "B".
• "newdata" will always refer to the other graph
• Handle "graph switching" in dotruralsepake.store.connect()
• Graph switching is based on A's and B's maximal timestamp for sepake:graphSetAsDefault
• Delete index as part of reindexing
• Only delete graph before seed is read
• Figure out a sensible "tie handler" if timestamps are equal or one or both are missing.

[mr-niels-christensen]

TODO: Tools for transition/cleaning up graphs