Failed to add/update records when using MongoDB backend

[dyj6146]

When I submit tasks to either direct or load balanced view using either map or apply_sync I see debug logs below:

2018-09-18 23:27:05.936 [IPClusterStart] 2018-09-18 23:27:05.935 [IPControllerApp] ERROR | DB Error updating record u'e5925b97-b0906f91a3415ce89c3c2299'
2018-09-18 23:27:05.936 [IPClusterStart] Traceback (most recent call last):
2018-09-18 23:27:05.937 [IPClusterStart]   File "/home/ifu/anaconda2/lib/python2.7/site-packages/ipyparallel/controller/hub.py", line 693, in save_queue_result
2018-09-18 23:27:05.937 [IPClusterStart]     self.db.update_record(msg_id, result)
2018-09-18 23:27:05.937 [IPClusterStart]   File "/home/ifu/anaconda2/lib/python2.7/site-packages/ipyparallel/controller/mongodb.py", line 87, in update_record
2018-09-18 23:27:05.937 [IPClusterStart]     rec = self._binary_buffers(rec)
2018-09-18 23:27:05.937 [IPClusterStart]   File "/home/ifu/anaconda2/lib/python2.7/site-packages/ipyparallel/controller/mongodb.py", line 68, in _binary_buffers
2018-09-18 23:27:05.937 [IPClusterStart]     rec[key] = list(map(Binary, rec[key]))
2018-09-18 23:27:05.937 [IPClusterStart]   File "/home/ifu/anaconda2/lib/python2.7/site-packages/bson/binary.py", line 147, in __new__
2018-09-18 23:27:05.937 [IPClusterStart]     raise TypeError("data must be an instance of bytes")
2018-09-18 23:27:05.937 [IPClusterStart] TypeError: data must be an instance of bytes

Looks like rec['buffers'] cannot be binarized?

I am running on ubuntu 17.04 with anaconda python 2.7.15, ipyparallel 6.2.2 and pymongo 3.7.0.

[dyj6146]

Here is a sample record on adding to the collection (by uncommenting print rec in add_record()):

2018-09-19 00:22:04.126 [IPClusterStart] {'result_header': None, 'stdout': '', 'msg_id': u'05a53bc7-ae912645c2b66b19baff4d42', 'result_buffers': None, 'buffers': [<memory at 0x7f3162f192b0>, <memory at 0x7f3162f19218>], 'execute_input': None, 'result_metadata': None, 'header': {u'username': u'ifu', u'version': u'5.3', u'msg_type': u'apply_request', u'msg_id': u'05a53bc7-ae912645c2b66b19baff4d42', u'session': u'dbd38529-ec967b7b98e819281a04ae8b', u'date': datetime.datetime(2018, 9, 18, 16, 22, 4, 97580, tzinfo=tzutc())}, 'content': {}, 'metadata': {}, 'started': None, 'completed': None, 'result_content': None, 'execute_result': None, 'engine_uuid': u'446d4291-0dc058b3c432b2724a8019d1', 'received': None, 'resubmitted': None, 'client_uuid': u'dbd38529-ec967b7b98e819281a04ae8b', 'submitted': datetime.datetime(2018, 9, 18, 16, 22, 4, 97580, tzinfo=tzutc()), 'queue': 'mux', 'stderr': '', 'error': None}

[minrk]

Looks like you're right that pymongo doesn't support the Python buffer interface and we should be casting memoryviews to bytes.