catalog 500 error on some datasets in block 'collection_resources'

[FuhuXia]

Some dataset returns 500 error with error message

ERR {{ super() }}
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckan/templates/page.html", line 19, in block 'page'
ERR {%- block content %}
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckan/templates/page.html", line 22, in block 'content'
ERR {% block main_content %}
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckan/templates/page.html", line 74, in block 'main_content'
ERR {% block primary %}
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckan/templates/page.html", line 87, in block 'primary'
ERR {% block primary_content %}
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckanext/datagovtheme/templates/package/read_base.html", line 59, in block 'primary_content'
ERR {% block primary_content_inner %}{% endblock %}
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckan/templates/package/read.html", line 34, in block 'primary_content_inner'
ERR {% block package_resources %}
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckanext/datagovtheme/templates/package/read.html", line 113, in block 'package_resources'
ERR {% block collection_resources %}
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckanext/datagovtheme/templates/package/read.html", line 114, in block 'collection_resources'
ERR {{ super() }}
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckanext/geodatagov/templates/package/read.html", line 14, in block 'collection_resources'
ERR {% set collection_package = h.get_collection_package(collection_package_id) %}
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckanext/geodatagov/helpers.py", line 62, in get_collection_package
ERR package = p.toolkit.get_action('package_show')({}, {'id': collection_package_id})
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckan/logic/__init__.py", line 551, in wrapped
ERR result = _action(context, data_dict, **kw)
ERR File "/home/vcap/deps/1/python/lib/python3.9/site-packages/ckan/logic/action/get.py", line 1018, in package_show
ERR raise NotFound
ERR ckan.logic.NotFound
OUT 2023-12-11 19:20:31,334 INFO  [ckan.config.middleware.flask_app]  500 /dataset/tiger-line-shapefile-2013-county-santa-cruz-county-ca-all-roads-county-based-shapefile render time 4.394 seconds

How to reproduce

1. Description of steps to reproduce the issue. Go to this dataset url, or search " 500 " NginxLog in NewRelic on app catalog-proxy for more examples.

Expected behavior

200 or 404 response code

Actual behavior

500 response code

Sketch

[Notes or a checklist reflecting our understanding of the selected approach]

[gujral-rei]

Handle 500 error gracefully

[btylerburton]

Previously there was a 500 graceful error page. That was lost during the cloud.gov migration. Possibly tied to NGINX config.

[chris-macdermaid]

Here's a screenshot of what I'm seeing.

[Jin-Sun-tts]

checked those datasets are in a collection, but the parent is deleted.

[Jin-Sun-tts]

FYI. The UI display for datasets have valid parent collection:

The error returned when the parent collection was not found. To address this, we implemented error handling and have the following options for UI changes for this situation:

1. Maintain the current display format but include a message indicating that the parent collection does not exist.

2. Enhance the display by adding a link to search for sibling datasets within the same collection.

3. Completely omit the 'Collection' section from the display in this scenario.

Furthermore, will do additional research to understand how the harvester behaves when harvesting dataset without a parent collection existing.

[Jin-Sun-tts]

After discussing with the team, we have decided to resolve the 500 error by implementing the following UI changes:

Additionally, created a new ticket (https://github.com/GSA/data.gov/issues/4594) to implement the necessary logic to prevent the deletion of the parent collection during the harvesting stage.

[Jin-Sun-tts]

After pushed the change to catalog, double checked on catalog production, the error from the missing parent issue is fixed.