geopython / pycsw

pycsw is an OGC CSW server implementation written in Python. pycsw fully implements the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web]. Initial development started in 2010 (more formally announced in 2011). The project is certified OGC Compliant, and is an OGC Reference Implementation. pycsw allows for the publishing and discovery of geospatial metadata via numerous APIs (CSW 2/CSW 3, OpenSearch, OAI-PMH, SRU). Existing repositories of geospatial metadata can also be exposed, providing a standards-based metadata and catalogue component of spatial data infrastructures. pycsw is Open Source, released under an MIT license, and runs on all major platforms (Windows, Linux, Mac OS X). Please read the docs at https://pycsw.org/docs for more information.
https://pycsw.org
MIT License
203 stars 155 forks source link

Harvest fails when trying to insert record with long abstract #470

Closed igor-chernikov closed 1 month ago

igor-chernikov commented 7 years ago

Description

Inserting record with long abstract causes the error:

ExceptionReport: 'Harvest (insert) failed: ERROR: index row size 2984 exceeds maximum 2712 for index "ix_records_abstract"\nHINT: Values larger than 1/3 of a buffer page cannot be indexed.\nConsider a function index of an MD5 hash of the value, or use full text indexing.\n.'

Sample record: http://geobretagne.fr/geonetwork/srv/eng/csw?service=CSW&version=2.0.2&request=GetRecordById&id=b5cb375d-7bb5-4b75-84f7-649e8f84e0dd

Environment

Sample code using owslib:

from owslib import csw
client = csw.CatalogueServiceWeb('http://localhost:9000/')  # pycsw url
client.harvest('http://geobretagne.fr/geonetwork/srv/eng/csw', 'http://www.opengis.net/cat/csw/2.0.2')

Additional Information

kalxas commented 7 years ago

@igor-chernikov thanks for the report, can you please include the database information?

igor-chernikov commented 7 years ago

@kalxas, sure, excuse me for not mentioning that earlier. I'm using PostgreSQL 9.5 with PostGIS 2.2

kalxas commented 7 years ago

It seems that you hit the index limit for the specific field. A workaround would be to drop that index completely. If you have full text search enabled, you will still be able to search your catalog through free text search.

igor-chernikov commented 7 years ago

@kalxas thanks for your response. I used exactly this workaround.