gramps-project / gramps-web-api

A RESTful web API for Gramps
GNU Affero General Public License v3.0
77 stars 40 forks source link

Server fails (code 500) when private families are in DB #523

Open ant20879g opened 1 month ago

ant20879g commented 1 month ago

Gramps 5.2.2 Gramps Web API 2.3.1 Gramps Web Frontend 24.6.0 Gramps QL 0.3.0 locale: en multi-tree: false task queue: true

GET /api/families/?locale=en&profile=self&keys=gramps_id,profile,change&page=2&pagesize=20) I marked few families as private, there're more than 20 non-families. Based on default filter all private families are at the end of the list or on the second page. GET works for first page and crashes for second one.

DavidMStraub commented 1 month ago

It can't be because of families being private, there must be another reason.

Please check the server logs and report them here. 500 always means a Python exception is thrown.

DavidMStraub commented 1 month ago

Please also try whether the problem persists after running check & repair on your database.

ant20879g commented 1 month ago

.... some lines are skipped for clarity.... grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/webargs/core.py", line 649, in wrapper grampsweb-1 | return func(*args, **kwargs) grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/base.py", line 446, in get grampsweb-1 | [self.full_object(obj, args, locale=locale) for obj in objects], grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/base.py", line 446, in grampsweb-1 | [self.full_object(obj, args, locale=locale) for obj in objects], grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/base.py", line 93, in full_object grampsweb-1 | obj = self.object_extend(obj, args, locale=locale) grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/families.py", line 55, in object_extend grampsweb-1 | obj.profile = get_family_profile_for_object( grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 505, in get_family_profile_for_object grampsweb-1 | "father": get_person_profile_for_handle( grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 473, in get_person_profile_for_handle grampsweb-1 | return get_person_profile_for_object(db_handle, obj, args, locale=locale) grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 399, in get_person_profile_for_object grampsweb-1 | birth, birth_event = get_birth_profile( grampsweb-1 | ^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 266, in get_birth_profile grampsweb-1 | event = get_birth_or_fallback(db_handle, person) grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps/gen/utils/db.py", line 59, in get_birth_or_fallback grampsweb-1 | birth_ref = person.get_birth_ref() grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | AttributeError: 'NoneType' object has no attribute 'get_birth_ref'

This happens when public family has private parents. gramps lib'ssanitize_family() does not set parent handle when it's private, so family with null in one or two handles is legitimate case. For this particular case this line better to be in try/finally block.

ant20879g commented 1 month ago

When I make family private (for private parents) I'm getting slightly different error:

...some lines are skipped for clarity... grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/webargs/core.py", line 649, in wrapper grampsweb-1 | return func(*args, **kwargs) grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/base.py", line 446, in get grampsweb-1 | [self.full_object(obj, args, locale=locale) for obj in objects], grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/base.py", line 446, in grampsweb-1 | [self.full_object(obj, args, locale=locale) for obj in objects], grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/base.py", line 93, in full_object grampsweb-1 | obj = self.object_extend(obj, args, locale=locale) grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/families.py", line 55, in object_extend grampsweb-1 | obj.profile = get_family_profile_for_object( grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 514, in get_family_profile_for_object grampsweb-1 | "children": [ grampsweb-1 | ^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 515, in grampsweb-1 | get_person_profile_for_handle( grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 473, in get_person_profile_for_handle grampsweb-1 | return get_person_profile_for_object(db_handle, obj, args, locale=locale) grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 399, in get_person_profile_for_object grampsweb-1 | birth, birth_event = get_birth_profile( grampsweb-1 | ^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 266, in get_birth_profile grampsweb-1 | event = get_birth_or_fallback(db_handle, person) grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | File "/usr/local/lib/python3.11/dist-packages/gramps/gen/utils/db.py", line 59, in get_birth_or_fallback grampsweb-1 | birth_ref = person.get_birth_ref() grampsweb-1 | ^^^^^^^^^^^^^^^^^^^^ grampsweb-1 | AttributeError: 'NoneType' object has no attribute 'get_birth_ref'

I do not see families with private children in my DB, however, at the end it still crashes when get_person_profile_for_object() is called with null in the second parameter.

ant20879g commented 1 month ago

Forgot to mention before, DB check did not find any errors.

DavidMStraub commented 1 month ago

Thanks! I see the problem – Gramps' PrivateProxyDb returns None if a person is private rather than HandleError :fearful: Why, actually?

Nick-Hall commented 1 month ago

The return value of None indicates that the handle exists, but has been filtered out by the proxy. The HandleError exception is a real error - the handle is referenced by an object, but doesn't actually exist.

In the past we used None for both cases, but this made tracking down errors very difficult and we had a large number of reports of associated NoneType errors.

DavidMStraub commented 1 month ago

and we had a large number of reports of associated NoneType errors.

Such a great argument in favour of type hints :wink:

Nick-Hall commented 1 month ago

The main problem was that we didn't raise an exception for the errors. They just slipped through looking like proxy filtering.