search

GeoNode / geonode

GeoNode is an open source platform that facilitates the creation, sharing, and collaborative use of geospatial data.
https://geonode.org/
Other
1.46k stars 1.13k forks source link

500 error when page_size is less than resource count #12712

Closed jeff-bradley closed 1 week ago

jeff-bradley commented 2 weeks ago

Expected Behavior

Have the api not return a 500 error when number of resources is less than page_size value

Actual Behavior

When scrolling to a 2nd page on All resources it causes the page to load "Page Loading Error The selected resources page is not available" it seems to be connected with page_size, if I change to page_size=3 (or a value less than the resource count) i get a successful response.

I initially was seeing this error on page=1 when I had less than 24 resources and then adding more than 24 caused a 200 response.

I also am seeing this error with the Documents page on the initial load of page=1 and I am unable to see any documents returned. I have less than 24 documents total.

The Dataset, Geostories and Dashboard view all load fine even with less than 24 resources.

Here is the debug response error DoesNotExist at /api/v2/resources Document matching query does not exist. Request Method: GET Request URL: {myhttpsdomain}/api/v2/resources?api_preset=catalog_list&filter%7Bmetadata_only%7D=false&page=2&page_size=24 Django Version: 4.2.9 Exception Type: DoesNotExist Exception Value:
Document matching query does not exist. Exception Location: /usr/local/lib/python3.10/dist-packages/django/db/models/query.py, line 637, in get Raised during: geonode.base.api.views.ResourceBaseViewSet Python Executable: /usr/local/bin/uwsgi Python Version: 3.10.12

Steps to Reproduce the Problem

  1. Add layers to Geonode
  2. Click All resources
  3. Scroll down to make a 2nd page load

Specifications

mattiagiupponi commented 2 weeks ago

can you please provide the geonode.log? I was not able to reproduce it locally you can find them in the django container under /var/log/geonode.log

jeff-bradley commented 2 weeks ago

Hello @mattiagiupponi Here is what seems to be relevant in the /var/log/geonode.log file

[pid: 445|app: 0|req: 12/247] 172.18.0.6 () {78 vars in 2521 bytes} [Tue Nov 12 03:13:58 2024] POST /account/login/ => generated 0 bytes in 78 msecs (HTTP/1.0 302) 16 headers in 956 bytes (1 switches on core 0)
[pid: 446|app: 0|req: 12/248] 172.18.0.6 () {74 vars in 3130 bytes} [Tue Nov 12 03:13:58 2024] GET /catalogue/ => generated 67284 bytes in 131 msecs (HTTP/1.0 200) 11 headers in 542 bytes (1 switches on core 0)
[pid: 449|app: 0|req: 12/249] 172.18.0.6 () {70 vars in 2413 bytes} [Tue Nov 12 03:13:59 2024] GET /api/v2/ => generated 1331 bytes in 38 msecs (HTTP/1.0 200) 11 headers in 432 bytes (1 switches on core 0)
[pid: 448|app: 0|req: 12/250] 172.18.0.6 () {70 vars in 2433 bytes} [Tue Nov 12 03:13:59 2024] GET /api/o/v4/userinfo => generated 288 bytes in 44 msecs (HTTP/1.0 200) 12 headers in 439 bytes (1 switches on core 0)
[pid: 447|app: 0|req: 12/251] 172.18.0.6 () {70 vars in 2434 bytes} [Tue Nov 12 03:13:59 2024] GET /api/v2/users/1000 => generated 318 bytes in 60 msecs (HTTP/1.0 200) 11 headers in 431 bytes (1 switches on core 0)
[pid: 450|app: 0|req: 12/252] 172.18.0.8 () {32 vars in 555 bytes} [Tue Nov 12 03:14:00 2024] POST /api/o/v4/tokeninfo/ => generated 249 bytes in 40 msecs (HTTP/1.1 200) 10 headers in 503 bytes (1 switches on core 0)
[pid: 451|app: 0|req: 11/253] 172.18.0.8 () {26 vars in 357 bytes} [Tue Nov 12 03:14:00 2024] GET /api/users/admin => generated 148 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 449 bytes (1 switches on core 0)
[pid: 452|app: 0|req: 12/254] 172.18.0.8 () {26 vars in 345 bytes} [Tue Nov 12 03:14:00 2024] GET /api/roles => generated 165 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 449 bytes (1 switches on core 0)
[pid: 445|app: 0|req: 13/255] 172.18.0.6 () {72 vars in 2641 bytes} [Tue Nov 12 03:14:00 2024] GET /api/v2/resources?api_preset=catalog_list&filter%7Bmetadata_only%7D=false&page=1&page_size=24 => generated 25851 bytes in 2260 msecs (HTTP/1.0 200) 11 headers in 454 bytes (1 switches on core 0)
Internal Server Error: /api/v2/resources
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.10/dist-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/usr/local/lib/python3.10/dist-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/usr/local/lib/python3.10/dist-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/usr/src/geonode/geonode/base/api/mixins.py", line 52, in list
    return self.get_paginated_response(serializer.data)
  File "/usr/local/lib/python3.10/dist-packages/dynamic_rest/bases.py", line 23, in wrapper
    self._resettable_cached_properties[func.__name__] = func(self)
  File "/usr/local/lib/python3.10/dist-packages/dynamic_rest/serializers.py", line 91, in data
    data = super(DynamicListSerializer, self).data
  File "/usr/local/lib/python3.10/dist-packages/rest_framework/serializers.py", line 768, in data
    ret = super().data
  File "/usr/local/lib/python3.10/dist-packages/rest_framework/serializers.py", line 253, in data
    self._data = self.to_representation(self.instance)
  File "/usr/local/lib/python3.10/dist-packages/dynamic_rest/serializers.py", line 70, in to_representation
    return [self.child.to_representation(item) for item in iterable]
  File "/usr/local/lib/python3.10/dist-packages/dynamic_rest/serializers.py", line 70, in <listcomp>
    return [self.child.to_representation(item) for item in iterable]
  File "/usr/local/lib/python3.10/dist-packages/dynamic_rest/serializers.py", line 704, in to_representation
    self.obj_cache[pk] = self._to_representation(instance)
  File "/usr/local/lib/python3.10/dist-packages/dynamic_rest/serializers.py", line 654, in _to_representation
    representation = self._faster_to_representation(instance)
  File "/usr/local/lib/python3.10/dist-packages/dynamic_rest/serializers.py", line 640, in _faster_to_representation
    ret[field.field_name] = field.to_representation(attribute)
  File "/usr/local/lib/python3.10/dist-packages/dynamic_rest/fields/fields.py", line 319, in to_representation
    return serializer.to_representation(instance)
  File "/usr/src/geonode/geonode/base/api/serializers.py", line 526, in to_representation
    return resource.get_user_perms(request.user) if request and request.user and resource else []
  File "/usr/src/geonode/geonode/security/models.py", line 426, in get_user_perms
    perms = perms.union(calculate_perms(self.get_real_instance(), user))
  File "/usr/local/lib/python3.10/dist-packages/polymorphic/models.py", line 173, in get_real_instance
    return real_model.objects.db_manager(self._state.db).get(pk=self.pk)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
geonode.documents.models.Document.DoesNotExist: Document matching query does not exist.
[pid: 446|app: 0|req: 13/256] 172.18.0.6 () {72 vars in 2641 bytes} [Tue Nov 12 03:14:09 2024] GET /api/v2/resources?api_preset=catalog_list&filter%7Bmetadata_only%7D=false&page=2&page_size=24 => generated 288773 bytes in 625 msecs (HTTP/1.0 500) 10 headers in 426 bytes (1 switches on core 0)
[pid: 449|app: 0|req: 13/257] 172.18.0.4 () {24 vars in 252 bytes} [Tue Nov 12 03:14:11 2024] GET / => generated 0 bytes in 1 msecs (HTTP/1.1 302) 9 headers in 347 bytes (1 switches on core 0)
jeff-bradley commented 1 week ago

I believe it has been fixed, the issue looked to be from doing a restore from an older version where 2 of the documents were listed in the resourcebase table but there was no actual document that was loaded in the restore. After removing the 2 entries from the base_resourcebase and corresponding, base_link and base_contract tables, both the All resources and Documents views work now. Thank you for the help!