Logging debug fails when pass iterator as ids in index task

[toopy]

Here ids[0], ids[-1], len(ids) can generate TypeError.

It happens when you pass something like YourModel.objects.values_list('id', flat=True) as ids parameter, but it can be very convenient:

https://github.com/mozilla/elasticutils/blob/master/elasticutils/contrib/django/tasks.py#L46

[willkg]

What does the traceback look like?

[toopy]

It can be obtained from this kind of code:

class MyMappingType(MappingType, Indexable):
    def reindex_all(cls):
        tasks.index_objects.delay(cls, cls.get_model().objects.values_list('id', flat=True), id_field=cls.id_field)

Here is the full traceback:

(myapp)florent@florent:~/Work/myapp/myapp$ myapp reindex_all
Traceback (most recent call last):
  File "/home/florent/.virtualenvs/myapp/bin/myapp", line 9, in <module>
    load_entry_point('myapp==0.1dev', 'console_scripts', 'myapp')()
  File "/home/florent/Work/myapp/myapp/myapp/manage.py", line 10, in main
    execute_from_command_line(sys.argv)
  File "/home/florent/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/home/florent/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/florent/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/florent/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/home/florent/Work/myapp/myapp/myapp/management/commands/reindex_all.py", line 21, in handle
    m_type.run_index_all()
  File "/home/florent/Work/myapp/django-esutils/django_esutils/mappings.py", line 194, in run_index_all
    flat=True))
  File "/home/florent/Work/myapp/django-esutils/django_esutils/mappings.py", line 189, in run_index
    tasks.index_objects.delay(cls, ids, id_field=cls.id_field)
  File "/home/florent/.virtualenvs/myapp/local/lib/python2.7/site-packages/celery/app/task.py", line 453, in delay
    return self.apply_async(args, kwargs)
  File "/home/florent/.virtualenvs/myapp/local/lib/python2.7/site-packages/celery/app/task.py", line 547, in apply_async
    link=link, link_error=link_error, **options)
  File "/home/florent/.virtualenvs/myapp/local/lib/python2.7/site-packages/celery/app/task.py", line 739, in apply
    request=request, propagate=throw)
  File "/home/florent/.virtualenvs/myapp/local/lib/python2.7/site-packages/celery/app/trace.py", line 354, in eager_trace_task
    uuid, args, kwargs, request)
  File "/home/florent/.virtualenvs/myapp/local/lib/python2.7/site-packages/celery/app/trace.py", line 253, in trace_task
    I, R, state, retval = on_error(task_request, exc, uuid)
  File "/home/florent/.virtualenvs/myapp/local/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/florent/Work/myapp/elasticutils/elasticutils/contrib/django/tasks.py", line 48, in index_objects
    ids[0], ids[-1], len(ids)))
  File "/home/florent/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/db/models/query.py", line 112, in __getitem__
    "Negative indexing is not supported."
AssertionError: Negative indexing is not supported.

[willkg]

It looks like you're passing an iterator as a task argument. I really wouldn't do this. I'd pass either a list or a tuple and not an iterator. Easy way to do this is to wrap your queryset in a list() call.

[toopy]

According that, in real life, the index task is running asynchronously, it's clearly dangerous. I close.