philipbl / python-persistent-queue

A persistent queue for Python
9 stars 4 forks source link

bson error #14

Closed Kriechi closed 7 years ago

Kriechi commented 7 years ago

BSON seems to fail at some point. Try to run this code:

from persistent_queue import PersistentQueue
import bson

queue = PersistentQueue('test.queue', loads=bson.loads, dumps=bson.dumps)
queue.put(1)

It fails with the following error:

Traceback (most recent call last):
  File "bson_test.py", line 5, in <module>
    queue.put(1)
  File ".../python-persistent-queue/persistent_queue/persistent_queue.py", line 242, in put
    write_data(i)
  File ".../python-persistent-queue/persistent_queue/persistent_queue.py", line 202, in write_data
    data = self.dumps(item)
  File ".../python-persistent-queue/venv/lib/python3.5/site-packages/bson/__init__.py", line 38, in dumps
    return encode_document(obj, [], generator_func=generator, on_unknown=on_unknown)
  File ".../python-persistent-queue/venv/lib/python3.5/site-packages/bson/codec.py", line 241, in encode_document
    key_iter = iterkeys(obj)
  File ".../python-persistent-queue/venv/lib/python3.5/site-packages/six.py", line 575, in iterkeys
    return iter(d.keys(**kw))
AttributeError: 'int' object has no attribute 'keys'
Kriechi commented 7 years ago

Since this seems to be a general issue with BSON, I would suggest to rephrase the README. Right now the README shows an example, and right below it, it says "has been tested with BSON". So this suggests that it should work - which it doesn't.

philipbl commented 7 years ago

BSON seems to only work with dictionaries (that's why I only do some of the tests with BSON). So to modify your example:

from persistent_queue import PersistentQueue
import bson

queue = PersistentQueue('test.queue', loads=bson.loads, dumps=bson.dumps)
queue.put({"foobar": 1})

Maybe that is too confusing and should be removed from the README?

Kriechi commented 7 years ago

Yes, the README should reflect this limitation of BSON - it is too confusing/unknown.

I would be nice if we could run all tests with bson - I guess we only have to change the items we put/get.