Batching discrete updates

[cayuu]

Regarding .body, the spec currently says:

However, when specifying .ids or other .match constraints, the .body field MUST be empty or contain only one element, and the action SHOULD apply the data element to .ids

This approach currently prevents batching multiple discrete updates in a single Qe. For example, if you wanted to update 2 users, setting one's skill field and the other's age field, while you might hope to do the following, it would be invalid according to the spec:

var qe = {
  do: 'update',
  on: 'users',
  ids: [1, 2],
  body: [ {skill:12}, {age:21} ]
}

The question is: do we want to support this "discrete update per id:body block"? Moreover: how could this be specified more clearly?

[cayuu]

First of all: specifying .ids with .body means we're in update action mode.

The states for this could be treated as:

	One .ids	Many .ids
One .body	:white_check_mark:	:white_check_mark:
Many .body	:no_entry_sign:	:large_blue_circle:

In more detail:

• One .ids and One .body - :white_check_mark: Apply the body to the specified ID
• Many .ids and One .body - :white_check_mark: Apply the body to all IDs
• One .ids and Many .body - :no_entry_sign: Not allowed - just collapse your bodies into one object
• Many .ids and Many .body - :large_blue_circle: ONLY if the number of IDs matches the number of Bodies - then apply updates as pairs eg: apply( id[0], body[0] ). In other words .ids.length !== .body.length is INVALID.

This final state (Many+Many if same number of elements) can be treated by a Qe implementation as allowing batching of discrete updates per declared id.

Some further questions:

• What happens if .match is also thrown in there (sub-select within .ids)?
• What happens if .update is also thrown in there (non-idempotent changes)?

[cayuu]

If .match is provided AT ALL, then only one body element is allowed (because the number of results is potentially unknown).

If .update is provided in addition to .body then the only question is one of priority, because .update files are ALL applied to EVERY result (by ids or match or a combination). It would seem to make sense to apply the idempotent change first, as applying it last would potentially overwrite any non-idempotent changes, eg:

{ do: 'update', on:'users', update:[ {points: {inc:5}} ], body:[ {points:20} ] }

Of course the obvious thing falling out of the example is that in situations where overwrite "may" occur, there exists sufficient knowledge to collapse the non-idempotent update into the idempotent write (ie set points directly to points:25).

Which means, order of operation should be irrelevant (if implementations are "doing the right thing").

Applying .body last however would make it clear that non-idempotent updates weren't applying in the case of clients "doing it wrong".

[cayuu]

Another view on states for action {on: 'update'}:

	One .body	Many .body
.match	:white_check_mark:	:no_entry_sign:
.update	:white_check_mark:	:no_entry_sign:
One .ids	:white_check_mark:	:no_entry_sign:
Many .ids	:white_check_mark:	:large_blue_circle: equal num

The left hand column shows presence of fields in the Qe. If the :no_entry_sign: symbol is showing, "many .body" is not permitted.

More simply: many .body elements are ONLY permitted when:

• .ids field is set to the same number of elements as .body,
• AND the .match and .update fields are NOT SET.

Where .update and .body are validly set, prefer to apply .update THEN .body, so that fields that could potentially be overwritten are set to the non-idempotent value of .body (makes it obvious that it's "doing it wrong").

[cayuu]

Closing this resolving:

(For "update" actions) Many .body elements are ONLY permitted when:

• .ids field is set to the same number of elements as .body,
• AND the .match and .update fields are NOT SET.

And:

If .body is provided and is modifying the same key on a record as the .update field, there exists sufficient knowledge to collapse the non-idempotent update into the idempotent write (ie. combine the update for that key into the .body field).

As such if both .update and .body are acting on the same record field, the Qe SHOULD return an error.

[cayuu]

And back in implementation land, collapsing discrete updates into a single control message is fraught with issues and edge cases. Reopening this with a recommendation to roll back the batch change (the update+body conflict resolution above is fine).