Read preference ignored on query

[samuelclay]

Steps to reproduce:

1) Create a row on a replica set that is experiencing slave lag (and they all do, even if it's just one second) 2) Query for that new row using read preference = PRIMARY 3) Half of the time the query comes back with the row not found, other times it's found. Using pymongo directly it works fine.

>>> settings.MONGODB.newsblur.fetch_history.find_one({'feed_id': 4100012}, read_preference=pymongo.ReadPreference.PRIMARY)
{u'feed_id': 4100012, u'_id': ObjectId('51a9907a82108812670286b7')}
>>> settings.MONGODB.newsblur.fetch_history.find_one({'feed_id': 4100012}, read_preference=pymongo.ReadPreference.PRIMARY)
{u'feed_id': 4100012, u'_id': ObjectId('51a9907a82108812670286b7')}
>>> settings.MONGODB.newsblur.fetch_history.find_one({'feed_id': 4100012}, read_preference=pymongo.ReadPreference.PRIMARY)
{u'feed_id': 4100012, u'_id': ObjectId('51a9907a82108812670286b7')}
>>> settings.MONGODB.newsblur.fetch_history.find_one({'feed_id': 4100012}, read_preference=pymongo.ReadPreference.PRIMARY)
{u'feed_id': 4100012, u'_id': ObjectId('51a9907a82108812670286b7')}
>>> settings.MONGODB.newsblur.fetch_history.find_one({'feed_id': 4100012}, read_preference=pymongo.ReadPreference.PRIMARY)
{u'feed_id': 4100012, u'_id': ObjectId('51a9907a82108812670286b7')}

And using Mongoengine's read preference:

>>> MFetchHistory.objects.get(feed_id=4100012, read_preference=pymongo.ReadPreference.PRIMARY)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/mongoengine-0.8.1-py2.7.egg/mongoengine/queryset/queryset.py", line 249, in get
    raise queryset._document.DoesNotExist(msg)
DoesNotExist: MFetchHistory matching query does not exist.
>>> MFetchHistory.objects.get(feed_id=4100012, read_preference=pymongo.ReadPreference.PRIMARY)
<MFetchHistory: MFetchHistory object>
>>> MFetchHistory.objects.get(feed_id=4100012, read_preference=pymongo.ReadPreference.PRIMARY)
<MFetchHistory: MFetchHistory object>
>>> MFetchHistory.objects.get(feed_id=4100012, read_preference=pymongo.ReadPreference.PRIMARY)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/mongoengine-0.8.1-py2.7.egg/mongoengine/queryset/queryset.py", line 249, in get
    raise queryset._document.DoesNotExist(msg)
DoesNotExist: MFetchHistory matching query does not exist.
>>> 

Notice that the pymongo call always works, but the mongoengine call fails half the time. It is probably using a read preference of PRIMARY_PREFERRED insead of PRIMARY, which causes it to not find the row.

[rozza]

thanks for the ticket @samuelclay this is an interesting one - because mongoengine just passes the read preference along unhindered (but just validated).

[rozza]

Ah found a bug! The method applying the read preference returns a clone and I failed to pass that clone on.

[rozza]

Fixed in master and will release in 0.8.2

In the meantime you can use:

MFetchHistory.objects.read_preference(pymongo.ReadPreference.PRIMARY).get(feed_id=4100012)

[samuelclay]

:tada: