search

ckan / ckanext-harvest

Remote harvesting extension for CKAN
130 stars 203 forks source link

Rebuilding search-index broken, when harvest errors exist. #528

Closed GordianDziwis closed 1 year ago

GordianDziwis commented 1 year ago

CKAN version

2.10

Describe the bug

Rebuilding search-index broken, when harvest errors exist in a job report.

Indexing dataset 5/962023-04-16 08:08:35,718 ERROR [ckan.lib.search] Error while indexing dataset 2bd0efb0-8d5c-4ff4-9603-2badecaf4599: TypeError('Object of type Row is not JSON serializable')
Object of type Row is not JSON serializable

It breaks here: https://github.com/ckan/ckan/blob/8aaf1c65b1a83d53296b71e16cfff4464694aad1/ckan/lib/search/index.py#L130

Steps to reproduce

  1. Add a harvester
  2. Harvest a source, which creates an error for a package
  3. ckan search-index rebuild
Zharktas commented 1 year ago

This is probably issue with ckanext-harvest itself, so i'm moving this to there.

seitenbau-govdata commented 1 year ago

We get the same error with CKAN 2.10. We will invest some time to have a deeper look to it. Our first guess is that it has something to do with the update of sqlalchemy from 1.3 to 1.4.