search

kartoza / SAEOSS-Portal

The SAEOSS Portal has been proposed as a system of software components functioning together as the national central earth observation geospatial repository, with a view to metadata and open geospatial API standards compliance as well as user impact maximization
https://kartoza.github.io/SAEOSS-Portal/
GNU Affero General Public License v3.0
4 stars 9 forks source link

Enhancing the ingestion endpoint #38

Open LunaAsefaw opened 1 year ago

EliVolsch commented 1 year ago

@LunaAsefaw Can you provide some clarity on this issue

EliVolsch commented 1 year ago

CSW or STAC dont want to harvest the entire catalog just certain parts / records from it.

zamuzakki commented 10 months ago

The harvester endpointis fixed in https://github.com/kartoza/SAEOSS-Portal/issues/36#issuecomment-1744048158 As for filtering harvest records, I saw that CSQL filter could be added. I'll look at this options.

zamuzakki commented 10 months ago

CSW or STAC dont want to harvest the entire catalog just certain parts / records from it.

To filter CSQ records, CQL can be used in Harvester, by adding it to Harvester configuration. Let's say, you have harvester source named emc-csw on local, you can go to http://localhost:5000/harvest/edit/emc-csw Then add CQL to configuration field image

In the above example, I only want to query CSW records with "Land Use" subject/tag. That configuraition will be translated to https://csw-testing.emc.kartoza.com/csw?service=CSW&version=2.0.2&resultType=results&startPosition=1&maxRecords=10&typeNames=csw:Record&request=GetRecords&typeNames=csw:Record&ElementSetName=full&constraintlanguage=CQL_TEXT&constraint=dc:subject%20like%20%Land%20Use%

More info about CQL can be found in: https://portal.ogc.org/files/96288#_requirements_class_features_filter

We can also add more configs, like specifying default tags. More infoon this: https://github.com/ckan/ckanext-harvest