Previously in #105 it is suggested that we prevent the orphaning of records. However, we need to be able to remove data from the system. At the very least it is often the case that someone might accidentally create a record that should not exist (e.g. someone just learning the system); these records should be removable and it is simply inexcusable that this system has no ability to support this basic action.
Barring the creation of an insert-only database as in #107, there ought to be a method for removing data. Ideally we ought to be able to regret our decision and recover seamlessly; that problem might be reserved for later, if here we only resolve to preserve the data required to make that action possible.
The method is simple: orphaning data leads to its removal. Something like:
The user makes an /update which results in some data being orphaned (that is, its parent is set to null, disconnecting it from the graph)
Magma examines the update and determines that orphaning will result.
It then calculates the total size of the orphaned subgraph(s) and replies to the user with the calculated result, perhaps a list of the orphaned graph ids and a confirmation hash computed from the input action. The update is parked somewhere.
The user replies to the update endpoint with the confirmation hash. Magma resumes the update and orphans the data.
Later, an orphaned data finder process comes along and collects the orphaned subgraph. It removes the records from the database and saves them to an archive table, perhaps keyed under ':::'.
Some problems:
We are assuming that (3) can be done efficiently. At present I'm not sure why it couldn't be, but if it is too slow, then what?
While orphaned data is hanging around waiting to be garbage collected, does it accidentally show up in queries? If so, how should it be kept out of them?
Previously in #105 it is suggested that we prevent the orphaning of records. However, we need to be able to remove data from the system. At the very least it is often the case that someone might accidentally create a record that should not exist (e.g. someone just learning the system); these records should be removable and it is simply inexcusable that this system has no ability to support this basic action.
Barring the creation of an insert-only database as in #107, there ought to be a method for removing data. Ideally we ought to be able to regret our decision and recover seamlessly; that problem might be reserved for later, if here we only resolve to preserve the data required to make that action possible.
The method is simple: orphaning data leads to its removal. Something like:
Some problems: