Binary fields cause user assertion

[jaraco]

We were using flashback in a troubleshooting session and we observed problems when using Flashback on MongoDB 2.6 when saving binary fields.

The error message we see in the logs is this:

2014-12-19T13:08:17.404-0800 [conn69] User Assertion: 52:The dollar ($) prefixed field '$binary' in 'enc_key.$binary' is not valid for storage.

I suspect though haven't confirmed that this assertion was triggered by FlashBack when it attempted to save a query which included a binary field.

I'm reporting this now for consideration and transparency, but I won't be able to troubleshoot further. I invite others to do so.

[jaraco]

I don't see anywhere in the code where flashback accounts for a $ in a query, which betrays my expectation.

[tredman]

Hi Jason - thanks for the report. Can you provide a sample of the op you were replaying? I don't need the actual query content but something representative of the structure would help.

Thanks, Travis

[jaraco]

Hi Travis. I'm not sure the exact op, but it was almost certainly an update containing a binary field, generated in Pymongo using something like:

import bson
spec = {'name': 'foo'}
doc = {'name': 'foo', 'enc_key': bson.Binary('\x00')}
coll.update(spec, doc, upsert=True)

@bdeeney might have more detail to share.

[bdeeney]

Hi Travis,

This is what one of the queries looks like in the oplog:

{"updateobj": {"enc_key": {"$binary": "EzZCY0O...", "$type": "00"}}, "query": {"_id": {"$oid": "5491ef809da39400027c466e"}}, "ns": "mydb.mycoll", "ts": {"$date": 1418850254073}, "op": "update"}

Please see #9 for a proposed solution. Thanks.

[tredman]

We found a robust way of handling this special type conversion in the mongo-tools library. This issue should be cleared up now in the latest commit. See https://github.com/ParsePlatform/flashback/issues/11 for details. I'm going to close this but please reopen or comment if you are still seeing problems.