gojomo
commented
10 years ago Interestingly enough, wrapping the operation with explicit transaction-management seems to help.
For example, while this in a django ./manage.py shell results in the runaway memory that motivates server-side-cursors:
items = MyModel.objects.all().order_by('pk').iterator()
for item in items:
print(item)
if item.pk > 2:
break..and this results in the above-reported error...
with server_side_cursors():
items = MyModel.objects.all().order_by('pk').iterator()
for item in items:
print(item)
if item.pk > 2:
break...this further refinement results in the desired behavior (only a reasonable number of models instantiated before exiting)...
with server_side_cursors():
with transaction.atomic():
items = MyModel.objects.all().order_by('pk').iterator()
for item in items:
print(item)
if item.pk > 2:
break(After competing some experiments, I did get other "Exception ignored" messages regarding "psycopg2.ProgrammingError: named cursor isn't valid anymore" later... but these seemed delayed/harmless.)
kurtwheeler
Tried using server_side_cursors() in Django 1.7; got the above error, on a line that tries to ensure a cursor is closed after another exception. (Line was added Jan-2014 in https://github.com/django/django/commit/0837eacc4e1fa7916e48135e8ba43f54a7a64997 - so suspect problem is new to Django 1.7.) Unsure if original exception was close-related.
Someone else trying to get server-side cursors working reported similar error at http://stackoverflow.com/questions/19069722/psycopg2-operationalerror-cursor-does-not-exist - resolution there suggests named cursors need not (and perhaps should not) be explicitly closed, because they're inherently transaction-scoped.