drf-haystack giving error 'invalid literal for int() with base 10:'

[heyitsjames]

I've created two other indexes which are working fine. I recently created a search index for my User model, with a UserSearchViewSet and UserSearchSerializer. Here's my code:

User model:

class User(AbstractBaseUser, PermissionsMixin):
    email = models.EmailField(max_length=255, unique=True)
    name = models.CharField(max_length=255, blank=True)
    token = models.CharField(max_length=200, default=None, blank=True, null=True)
    email_token = models.CharField(max_length=200, default=None, blank=True, null=True)
    new_email = models.EmailField( default=None, blank=True, null=True)
    is_staff = models.BooleanField(default=False)
    is_active = models.BooleanField(efault=True)
    notes = models.TextField(lank=True, null=True)
    network = models.ForeignKey(AdNetwork, related_name='users', blank=True, null=True)
    date_joined = models.DateTimeField(default=timezone.now)
    date_created = models.DateTimeField(auto_now_add=True, null=True, blank=True)
    last_updated = models.DateTimeField(auto_now=True)

User search index:

class UserIndex(indexes.SearchIndex, indexes.Indexable):
    text = indexes.CharField(document=True, use_template=True)
    network_id = indexes.CharField(model_attr='network_id')

    def get_model(self):
        return User

    def index_queryset(self, using=None):
        return self.get_model().objects.all()

User search ViewSet:

class UserSearchViewSet(HaystackViewSet):
    serializer_class = serializers.UserSearchSerializer
    index_models = [models.User]

    def get_queryset(self):
        user_network_id = str(self.request.user.network.id) # filter by user network id
        return super().get_queryset().filter(network_id=user_network_id)

User search Serializer:

class UserSearchSerializer(serializers.ModelSerializer, HaystackSerializerMixin):
    class Meta:
        model = User
        search_fields = ('text',)

So, here's the problem. Running ./manage.py rebuild_index works great, and I can see that the User models were indexed. However, going to the url users/search, I'm greeted with this error:

ValueError at /users/search/
invalid literal for int() with base 10: 'user.user.5'

Which seems to be occurring at rest_framework/fields.py line 837 in to_representation

My User search index and endpoint is quite literally a carbon copy of the other two I've created. DRF Haystack seems to be treating my User model different. Any ideas? I've been spinning my wheels for hours in the debugger with no luck.

[rhblind]

Hi,

It seems to me that you're casting the user.network.id to a str, while it really is an int? I doubt this has anything to do with drf-haystack or your user model, but is a type casting error somewhere.

Try setting your network_id to indexes.IntegerField(model_attr="network__id") and drop casting to str.

[heyitsjames]

@rhblind Casting to a string was a silly oversight, and I've corrected that, but the problem is on the user.id field, which, for some reason, in the to_representation function of DRF, refuses to turn into an integer, and instead is 'user.user.id'. I have two other models that have search indexes on them, using the exact same code, and they are working fine. Here's the stacktrace:

Environment:

Request Method: GET
Request URL: http://localhost:8000/users/search/

Django Version: 1.8.4
Python Version: 3.5.1
Installed Applications:
('django.contrib.auth',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.contenttypes',
 'ian.core',
 'ian.user',
 'ian.grid',
 'ian.push',
 'ian.site',
 'ian.plug',
 'flat',
 'django.contrib.admin',
 'kombu.transport.django',
 'djcelery',
 'rest_framework_swagger',
 'rest_framework',
 'corsheaders',
 'simple_history',
 'taggit',
 'taggit_serializer',
 'djrill',
 'djmail',
 'django_jinja',
 'django_jinja.contrib._humanize',
 'dbbackup',
 'django_extensions',
 'rolepermissions',
 'haystack')
Installed Middleware:
('corsheaders.middleware.CorsMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'django.middleware.security.SecurityMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'ian.grid.middleware.AdNetworkMiddleware',
 'simple_history.middleware.HistoryRequestMiddleware')

Traceback:
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/django/core/handlers/base.py" in get_response
  132.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/django/views/decorators/csrf.py" in wrapped_view
  58.         return view_func(*args, **kwargs)
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/rest_framework/viewsets.py" in view
  87.             return self.dispatch(request, *args, **kwargs)
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/rest_framework/views.py" in dispatch
  466.             response = self.handle_exception(exc)
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/rest_framework/views.py" in dispatch
  463.             response = handler(request, *args, **kwargs)
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/rest_framework/mixins.py" in list
  45.             return self.get_paginated_response(serializer.data)
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/rest_framework/serializers.py" in data
  658.         ret = super(ListSerializer, self).data
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/rest_framework/serializers.py" in data
  223.                 self._data = self.to_representation(self.instance)
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/rest_framework/serializers.py" in to_representation
  598.             self.child.to_representation(item) for item in iterable
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/rest_framework/serializers.py" in <listcomp>
  598.             self.child.to_representation(item) for item in iterable
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/rest_framework/serializers.py" in to_representation
  456.                 ret[field.field_name] = field.to_representation(attribute)
File "/Users/jamesrasmussen/.virtualenvs/ian-api/lib/python3.5/site-packages/rest_framework/fields.py" in to_representation
  837.         return int(value)

Exception Type: ValueError at /users/search/
Exception Value: invalid literal for int() with base 10: 'user.user.5'

Line 837 of fields.py is the IntegerField class's to_representation() function. Yes, it is a type casting error, but I cannot understand why the IntegerField is being supplied the string 'user.user.5', when the value should just be 5. This is where the type-casting error is occurring

[heyitsjames]

Figured the problem out, after painstakingly stepping through the debugger.

I had:

class UserSearchSerializer(UserSerializer, HaystackSerializerMixin):
    class Meta(UserSerializer.Meta):
        search_fields = ('text',)

When I should have had:

class UserSearchSerializer(HaystackSerializerMixin, UserSerializer):
    class Meta(UserSerializer.Meta):
        search_fields = ('text',)

Which used the to_representation of DRF's BaseSerializer, instead of HaystackSerializerMixin's to_representation, which returns instance.object, or the actual object at hand, instead of the SearchResult object. Yeesh.

[rhblind]

Good job! Glad you worked it out :)