search

Canadian-Geospatial-Platform / geocore

GeoCore is an Open Source Cloud Native (AWS) Geospatial Catalog | GeoCore est un catalogue géospatial Open Source Cloud Native (AWS)
https://canadian-geospatial-platform.github.io/geocore/
Other
2 stars 4 forks source link

Collection functionality is missing #19

Closed indiciumx closed 1 year ago

indiciumx commented 2 years ago

There's no collection (parent/child relationship) feature in Geo.ca. In FGP, a child record is related to its parent with the following tag:

08b810c2-7c81-40f1-adb1-c32c8a2c9f50 In FGP, having this tag bring buttons to access every child record from the parent record page: On the child record page, we have a button to access the parent and another button to display all the records in this collection in the search results: This function is really essential, otherwise, the parent records, which usually don't have any attached resources look really pointless since there's no reference to the components of the collection. A good example is "Floods in Canada - Cartographic Product Collection" which is featured on the Geo.ca front page. When you visit the record, there's no map, no resources and no mention of the datasets that make up the collection. https://app-stage.geo.ca/result?id=08b810c2-7c81-40f1-adb1-c32c8a2c9f50&lang=en The parent/child functionality is also missing in OpenMaps, so to go around that, we used to include some links in the description using some markdown language (not supported in geo.ca) to recreate the parent/child feature. Doing it this way once again even if there was support for markdown language would make it really confusing for the end user, so having buttons would be the way to go. [sc2.png](https://redmine.gcgeo.gc.ca/redmine/attachments/download/9003/sc2.png) (5.84 KB) Delete Martin Mimeault, 01/25/2022 02:21 PM [sc1.png](https://redmine.gcgeo.gc.ca/redmine/attachments/download/9004/sc1.png) (11.5 KB) Delete Martin Mimeault, 01/25/2022 02:21 PM [sc3.png](https://redmine.gcgeo.gc.ca/redmine/attachments/download/9005/sc3.png) (38.7 KB) Delete Martin Mimeault, 01/25/2022 02:21 PM [parentID.PNG](https://redmine.gcgeo.gc.ca/redmine/attachments/download/9203/parentID.PNG) (27.1 KB) Delete Bo Lu, 06/10/2022 01:34 PM
indiciumx commented 2 years ago

History #1Updated by Bo Lu 8 months ago Comment Edit Is there a way to support collections in GeoCore?

This record from DFO seems to be very boring in geo.ca :L

https://gcgeo.gc.ca/geonetwork/metadata/eng/4c7880ac-cb01-4e12-8305-2ae8eb675390

https://app-stage.geo.ca/result?id=4c7880ac-cb01-4e12-8305-2ae8eb675390&lang=en From #27495, here is a test case with uuid: 4c7880ac-cb01-4e12-8305-2ae8eb675390

#2Updated by Bo Lu 8 months ago Target version set to 1.0.5 #3Updated by Bo Lu 5 months ago Status changed from New to In Progress Comment Edit Steps to add this features: 1) Add code to the HNAP to GeoCore translation lambda to create a new field which contains the parent uuid. Currently, we are guaranteed that one child will only have one parent. 2) Generate the parquet file as normal, a new column will be created when the geocore format is normalized. Not work needed for this step. 3) Write a lambda function that contains a AWS Athena SQL function that queries a given uuid and returns child record(s) and a parent record if they exist as JSON via an API. 4) Front end programming is needed to read the JSON API and display the parent/child record(s).

#4Updated by Bo Lu 4 months ago File parentID.PNG added Comment Edit 1) adding parent identifier is complete

2) this is automatically generated 3) It is likely better to use a NoSQL database for storing this information. I.e., One lambda will read the parquet file and store the parent->child relationship in the NoSQL database. Another lambda will: 1) reveal what records are parents and their respective children (e.g., collections?summary=true) and 2) what are the children for a particular parent (e.g., collections?id=UUID)

#5Updated by Bo Lu 4 months ago Comment Edit After some final thoughts, I determined that it is easier and more dynamic to have a lambda read the parquet file and filter out the parents and siblings. Performance is good, <1 seconds.

Example usage: collections?uuid=9cad712a-5ac5-4248-b7d7-2db1a3892509

{ "statusCode": 200, "children_count": 3, "parent": { "id": "08b810c2-7c81-40f1-adb1-c32c8a2c9f50", "description_en": "Floods in Canada - Cartographic Product Collection", "description_fr": "Inondations au Canada - collections de produits cartographiques" }, "children": [ { "id": "74144824-206e-4cea-9fb9-72925a128189", "description_en": "Floods in Canada - Archive", "description_fr": "Inondations au Canada – Archive" }, { "id": "9cad712a-5ac5-4248-b7d7-2db1a3892509", "description_en": "Active Floods in Canada", "description_fr": "Inondations en cours au Canada" }, { "id": "b1afd8d2-6e14-4ec4-9a09-652221a6cb71", "description_en": "Floods in Canada - Current Year", "description_fr": "Inondations au Canada – Année courante" } ] } #6Updated by Bo Lu 3 months ago Comment Edit Changed 'uuid' to 'id' in the parameters for consistency between the search api. Displays 'self', which echoes the query parameter Example case 1, no parameters provided: collections?

{ "statusCode": 200, "body": "No id parameter was passed. Usage: ?id=XYZ" } Example case 2, searching for a non-existing record: collections?id=ASDF

{ "statusCode": 200, "sibling_count": 0, "self": null, "parent": null, "sibling": null } Example case 3, searching for a child record: collections?id=9cad712a-5ac5-4248-b7d7-2db1a3892509

{ "statusCode": 200, "sibling_count": 2, "self": { "id": "9cad712a-5ac5-4248-b7d7-2db1a3892509", "description_en": "Active Floods in Canada", "description_fr": "Inondations en cours au Canada" }, "parent": { "id": "08b810c2-7c81-40f1-adb1-c32c8a2c9f50", "description_en": "Floods in Canada - Cartographic Product Collection", "description_fr": "Inondations au Canada - collections de produits cartographiques" }, "sibling": [ { "id": "74144824-206e-4cea-9fb9-72925a128189", "description_en": "Floods in Canada - Archive", "description_fr": "Inondations au Canada – Archive" }, { "id": "b1afd8d2-6e14-4ec4-9a09-652221a6cb71", "description_en": "Floods in Canada - Current Year", "description_fr": "Inondations au Canada – Année courante" } ] } Example case 4, searching for a parent record: collections?id=08b810c2-7c81-40f1-adb1-c32c8a2c9f50

{ "statusCode": 200, "sibling_count": 3, "self": { "id": "08b810c2-7c81-40f1-adb1-c32c8a2c9f50", "description_en": "Floods in Canada - Cartographic Product Collection", "description_fr": "Inondations au Canada - collections de produits cartographiques" }, "parent": null, "sibling": [ { "id": "74144824-206e-4cea-9fb9-72925a128189", "description_en": "Floods in Canada - Archive", "description_fr": "Inondations au Canada – Archive" }, { "id": "9cad712a-5ac5-4248-b7d7-2db1a3892509", "description_en": "Active Floods in Canada", "description_fr": "Inondations en cours au Canada" }, { "id": "b1afd8d2-6e14-4ec4-9a09-652221a6cb71", "description_en": "Floods in Canada - Current Year", "description_fr": "Inondations au Canada – Année courante" } ] } Example case 5, searching for a record without sibling or parent: collections?id=2bcf34b5-4e9a-431b-9e43-1eace6c873bd

{ "statusCode": 200, "sibling_count": 0, "self": { "id": "2bcf34b5-4e9a-431b-9e43-1eace6c873bd", "description_en": "Inuit Communities Location", "description_fr": "Localisation des communautés inuites" }, "parent": null, "sibling": null } #7Updated by Bo Lu 3 months ago Status changed from In Progress to Resolved Comment Edit Marking as resolved.

Documentation available on: https://redmine.gcgeo.gc.ca/redmine/projects/geo-ca/wiki/Collections_API

#8Updated by Bo Lu 3 months ago % Done changed from 0 to 100

jvanulde commented 1 year ago

Pushed to PROD and functioning. Great work everyone! 🎉