restructure Catprez Endpoints and add annotation listings to root endpoint

recalcitrantsupplant commented 11 months ago

Root endpoint ("/") now includes:

the Prez version (when run directly with Python this will display as "0.1.0.dev0" .
the annotation predicates Prez will look for (for labels, descriptions, and provenance) to annotate data when the annotated turtle mediatype (text/anot+turtle) is requested. These are provided as an ordered list in terms of the priority a consumer should use them in if more than one predicate is present. This list is configurable via environment variables - see prez/config.
A list of endpoint information, example shown below:
```
endpoint:catalog-listing a ont:ListingEndpoint ;
ont:deliversClasses prez:CatalogList ;
ont:isTopLevelEndpoint "true"^^xsd:boolean ;
ont:baseClass dcat:Catalog ;
ont:endpointTemplate "/c/catalogs" ;
.
```
At this point the endpoints listed is not an exhaustive list - only the endpoints which deliver "instance data" have been included as this endpoint specification as RDF is used by the application.

The Catprez endpoints now follow the same structure as Spaceprez endpoints (with one less level of hierarchy): /c/catalogs /c/catalogs/{catalog_curie} /c/catalogs/{catalog_curie}/resources /c/catalogs/{catalog_curie}/resources/{resource_curie} This change was made to simplify frontend handling of data returned by Catprez. The catprez profile has been updated accordingly - Catalog items now display only a sample of their resources as part of the Catalog description - a full resource list can be viewed at the /c/catalogs/{catalog_curie}/resources endpoint

edmondchuc commented 11 months ago

Did an initial test of the PR with the new changes in prez-ui (from this PR https://github.com/RDFLib/prez-ui/pull/116) and it looks good!

I tested it with the ANU records, which have around 400+ in a catalog.

Only thing I'll comment on is the ordering of the items. Is there any ordering applied? Should we order it by label like we are doing with spaceprez?

recalcitrantsupplant commented 11 months ago

Did an initial test of the PR with the new changes in prez-ui (from this PR RDFLib/prez-ui#116) and it looks good!

I tested it with the ANU records, which have around 400+ in a catalog.

Only thing I'll comment on is the ordering of the items. Is there any ordering applied? Should we order it by label like we are doing with spaceprez?

ordering, no - I had a quick look at the ordering code and it is going to be difficult to adjust, suggest we revisit this - did you mean ordering for vocprez and not spaceprez? (though the query is the same)

edmondchuc commented 11 months ago

ordering, no - I had a quick look at the ordering code and it is going to be difficult to adjust, suggest we revisit this - did you mean ordering for vocprez and not spaceprez? (though the query is the same)

Ok, I thought it's using the same listing query, that's why I thought the ordering would be the same, but it seems like there's no ordering being applied at all.

Do you mind just elaborating a bit on why it's difficult and what are some of the issues?

recalcitrantsupplant commented 11 months ago

It is just using the listing query yes, back when you added the order by label it only looks at focus_item.top_level_listing - which covered the VocPrez use case (and might cover this one too) - but it should really be generalised - we'd just need to test it. The code block is:

{{
SELECT ?top_level_item ?child_item
WHERE {{
    {f'{uri_or_tl_item} a <{focus_item.base_class}> .{chr(10)}' if focus_item.top_level_listing else generate_focus_to_x_predicates(uri_or_tl_item, focus_to_child, focus_to_parent)}\

{f'''
    OPTIONAL {{
        {f'{uri_or_tl_item} <{ordering_predicate}> ?label .' if focus_item.top_level_listing else ""}
    }}
''' if settings.order_lists_by_label else ""}
}}
{f'''
{'ORDER BY ASC(?label)' if ordering_predicate else "ORDER BY ?top_level_item"}

ordering_predicate is at present the first predicate supplied in the settings.label_predicates (skos:prefLabel)

but... all of this can be simplified with sh:target/sh:select I proposed - you'd just write whatever subquery you want, and it would be inserted here. So I'm reluctant to spend too much time getting this implementation right.

Do you have a demo instance up somewhere we can run this against and see if it's an easy fix?

edmondchuc commented 11 months ago

@recalcitrantsupplant - yup, I've been testing it with the ANU catalog here https://api.anu.dev.kurrawong.ai/sparql. It has one catalog at the moment at https://anu.dev.kurrawong.ai/c/catalogs and around 400 records within it. The 400 records won't load in CatPrez right now because it's timing out but the data is available in the above SPARQL endpoint.

edmondchuc commented 11 months ago

It is just using the listing query yes

I guess I was just wondering how the spaceprez container items are being ordered but not the catprez implementation. You can see an example of the spaceprez stuff in the GSQ catalog here.

recalcitrantsupplant commented 11 months ago

This is the current query that gets generated for Spaceprez Datasets

PREFIX dcterms: <http://purl.org/dc/terms/>
PREFIX prez: <https://prez.dev/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>

CONSTRUCT {
  ?top_level_item a <http://www.w3.org/ns/dcat#Dataset> .
}
WHERE {
  ?top_level_item a <http://www.w3.org/ns/dcat#Dataset> .
  {
    SELECT ?top_level_item ?child_item
    WHERE {
      ?top_level_item a <http://www.w3.org/ns/dcat#Dataset> .
      OPTIONAL {
        ?top_level_item <http://www.w3.org/2004/02/skos/core#prefLabel> ?label .
      }
    }
    ORDER BY ASC(?label)
    LIMIT 20
    OFFSET 0
  }
}

And for Feature Collections, non top level, no ordering matches the conditionals in the query, but it may not be the desired behaviour..

PREFIX dcterms: <http://purl.org/dc/terms/>
PREFIX prez: <https://prez.dev/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>

CONSTRUCT {
  <https://linked.data.gov.au/dataset/qldgeofeatures> ?focus_to_child ?child_item .
}
WHERE {
  <https://linked.data.gov.au/dataset/qldgeofeatures> ?focus_to_child ?child_item .
  VALUES ?focus_to_child { <http://www.w3.org/2000/01/rdf-schema#member> }
  {
    SELECT ?top_level_item ?child_item
    WHERE {
      <https://linked.data.gov.au/dataset/qldgeofeatures> ?focus_to_child ?child_item .
      VALUES ?focus_to_child { <http://www.w3.org/2000/01/rdf-schema#member> }
      OPTIONAL {
      }
    }
    ORDER BY ASC(?label)
    LIMIT 20
    OFFSET 0
  }
}

RDFLib / prez

restructure Catprez Endpoints and add annotation listings to root endpoint #164