manage.py index doesn't work (for me)

[GoogleCodeExporter]

Here's my situation. I've installed djapian (2.3) with django (1.1),
postgresql (8.3) and python-psycopg2 and everything works as expected. I've
created my indexers, rebuild the index, and so on. Everything works except
when I try to update the index. When I run "manage.py index --verbose" one
of 2 things happen.
1.- I get a nice display saying: "There are N objects to update". But
nothing happens. The index doesn't get updated.
or 
2.- I get an error: django.db.transaction.TransactionManagementError:
Transaction managed block ended with pending COMMIT/ROLLBACK.

Can someone please let me know how to work around this?

Also, I can update the indexes doing something like this:

from app.models import MyModel
from common.indexes import MyModelIndex
MyModel.indexer.update()

After doing something like that I can search the changes I've made, but
"manage.py index" keeps saying there are objects to be changed.

Any help would be much appreciate.

FYI: I have python-xapian 1.0.7 and xapian-tools 1.0.7

Marcos

Original issue reported on code.google.com by marcosmo...@gmail.com on 24 Nov 2009 at 3:16

[GoogleCodeExporter]

I forgot to mention that I've tried this on two diferent linux distributions. 
Arch
and Ubuntu server. Both with the same results.

Original comment by marcosmo...@gmail.com on 24 Nov 2009 at 4:43

[GoogleCodeExporter]

Original comment by daevaorn on 24 Nov 2009 at 8:30

• Changed state: Accepted

[GoogleCodeExporter]

I have the same problem:

 * djapian (2.3)
 * django (trunk),
 * postgresql (8.4) 
 * python-psycopg2
 * Debian testing

My traceback is:

$ ./manage.py index --commit_each --verbose --per_page=1
There are 8770 objects to update
.Traceback (most recent call last):
  File "./manage.py", line 11, in <module>
    execute_manager(settings)
  File "/home/helder/python_local/django/core/management/__init__.py", line 439, in
execute_manager
    utility.execute()
  File "/home/helder/python_local/django/core/management/__init__.py", line 380, in
execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/helder/python_local/django/core/management/base.py", line 195, in
run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/helder/python_local/django/core/management/base.py", line 222, in execute
    output = self.handle(*args, **options)
  File "/arquivo/www/busca/newsrss/local/djapian/management/commands/index.py", line
153, in handle
    update_changes(verbose, timeout, not make_daemon, per_page, commit_each)
  File "/home/helder/python_local/django/db/transaction.py", line 267, in
_commit_manually
    leave_transaction_management()
  File "/home/helder/python_local/django/db/transaction.py", line 77, in
leave_transaction_management
    raise TransactionManagementError("Transaction managed block ended with pending
COMMIT/ROLLBACK")
django.db.transaction.TransactionManagementError: Transaction managed block 
ended
with pending COMMIT/ROLLBACK

When I try to do a MyModel.indexer.update() it seems that it tries to index all 
the
documents (this takes about an hour, so after a while I interrupt this process).

Helder

Original comment by hguerreiro@gmail.com on 27 Nov 2009 at 11:25

[GoogleCodeExporter]

I've created a patch for this issue. I would actually say it's three separate 
issues though:

1. With 2.3, the the transaction.commit() added per #26 was removed but the 
@transaction.commit_manually 
decorator was left. This means that the function will throw an exception if 
changes aren't properly committed. 
That's what @hguerreiro encountered.

I've added back the transaction.commit() line because that will commit any 
changes left uncommited before 
exiting the function. This line can be removed when all changes are always 
committed.

2. Since the transaction.commit() was removed, issue #26 should have been 
re-opened. With 2.3, running 
Djapian daemonized does not work propely as changes are not caught.

This issue is solved by the transaction.commit() line. However, when all 
changes are always committed, one 
could choose to run this line only when there are no changes.

3. There is no 'update' action. This is the reason @marcosmoyano keeps getting 
the "There are N objects to 
update" over and over again. When there are 'delete' or 'add' actions, he runs 
into issue 1 above.

I changed 'update' to 'edit'.

If you could add Djapian 2.2.4 to Pypi, I would be most grateful. I'm using 
Djapian with Buildout and 2.2.4 
worked fine for me but I can't define "Djapian==2.2.4" as a dependency anymore. 
Now I can only get 2.3, 
which is not working for me.

Actually, please leave old versions of Djapian on Pypi if you don't have to 
remove them. It makes defining 
specific versions of Djapian as a dependency much easier. Thanks!

Original comment by jonasnoc...@gmail.com on 3 Dec 2009 at 10:37

Attachments:

• editpatch.diff

[GoogleCodeExporter]

Original comment by daevaorn on 3 Dec 2009 at 11:30

• Added labels: Priority-High, Milestone-Release2.3
• Removed labels: Priority-Medium

[GoogleCodeExporter]

jonasnockert,
Your patch worked beautifully :-)
Thanks a lot!

Marcos

Original comment by marcosmo...@gmail.com on 4 Dec 2009 at 1:15

[GoogleCodeExporter]

The patch is OK, it worked like a charm.

Thanks!

Original comment by hguerreiro@gmail.com on 5 Dec 2009 at 2:20

[GoogleCodeExporter]

Fixed with r314, r315, r316.

Jonas, thanks again!

Original comment by daevaorn on 5 Dec 2009 at 9:16

• Changed state: Fixed

[GoogleCodeExporter]

Hello,

I get the same error but I'm already using the patched version 

Django          - 1.3          - active 
Djapian         - 2.3.1        - active

python manage.py index --rebuild succceds but python manage.py index gets this 
traceback:

File 
"/home/administrador/envs/pk2/lib/python2.6/site-packages/djapian/management/com
mands/index.py", line 166, in handle
    update_changes(verbose, timeout, not make_daemon, per_page, commit_each)
  File "/home/administrador/envs/pk2/lib/python2.6/site-packages/django/db/transaction.py", line 219, in inner
    self.__exit__(*sys.exc_info())
  File "/home/administrador/envs/pk2/lib/python2.6/site-packages/django/db/transaction.py", line 207, in __exit__
    self.exiting(exc_value, self.using)
  File "/home/administrador/envs/pk2/lib/python2.6/site-packages/django/db/transaction.py", line 302, in exiting
    leave_transaction_management(using=using)
  File "/home/administrador/envs/pk2/lib/python2.6/site-packages/django/db/transaction.py", line 56, in leave_transaction_management
    connection.leave_transaction_management()
  File "/home/administrador/envs/pk2/lib/python2.6/site-packages/django/db/backends/__init__.py", line 115, in leave_transaction_management
    raise TransactionManagementError("Transaction managed block ended with "
django.db.transaction.TransactionManagementError: Transaction managed block 
ended with pending COMMIT/ROLLBACK

I think this output has something to do with my population function but I don't 
know how:

@transaction.commit_manually
def save_register(ps):
    committed = True
    for i in ps:
        p = Package(**i)
        p.save()
        print i['name']
    else:
        committed = False
    if committed:
        transaction.commit()
    else:
        transaction.rollback()

The only place a model.save() is called is inside this function and the script 
that calls it works correctly.

So I don't know why rebuilding the index works but updating indexing doesn't. 
Before using save_register for population python manage.py index worked ok.

any tips?

Original comment by j.martin...@gmail.com on 11 May 2011 at 9:23

[GoogleCodeExporter]

The "Transaction managed block ended with pending COMMIT/ROLLBACK" error is a 
long-standing issue with Django's transaction.commit_manually decorator: 
https://code.djangoproject.com/ticket/6623 - if an uncaught exception is raised 
anywhere in the method, it will be masked by that error.

You can use this additional wrapper to find out the underlying exception: 
http://metak4ml.blogspot.com/2011/05/django-transactioncommitmanually-mask.html

In my case, the root cause was the bug detailed in issue #128.

Original comment by westdotc...@gmail.com on 31 Jul 2011 at 5:30