Describe isPartOf relations in DCAT-US dataset as Collection in catalog-next

FuhuXia commented 3 weeks ago

User Story

In order to show relations of DCAT-US datasets linked to each other with isPartOf field, data.gov teams wants group those datasets into Collections in the catalog-next.

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

Parent dataset behavior:

[ ] GIVEN a dataset's identifier is the value of isPartOf field of other datasets \ AND they are from the same harvest source \ WHEN the dataset is listed in a search result \ THEN a collection icon shows up to indicate it is a Collection \
[x] GIVEN a dataset's identifier is the value of isPartOf field of other datasets \ AND they are from the same harvest source \ AND all children datasets are in the deleted state \ WHEN the dataset is listed in a search result \ THEN there is NO collection icon to this dataset \
[x] GIVEN a dataset's identifier is the value of isPartOf field of other datasets \ AND they are from the same harvest source \ WHEN the dataset detail page is visited \ THEN there is a block to indicate this is a Collection \ AND there is a link to the Collection page.
[x] GIVEN a Collection page is visited \ THEN all the collection's children datasets are listed \ AND you can search within this Collection.

Children dataset behavior:

[x] GIVEN a dataset with a value for isPartOf value \ THEN the dataset is hidden from normal search result AND does not count in the total dataset count
[x] GIVEN a dataset with a value for isPartOf value \ THEN the dataset is listed when search with include_collection:true AND it is counted in the total dataset count
[x] GIVEN a dataset with a value for isPartOf value \ THEN the dataset is listed in its Collection page
[x] GIVEN a dataset with a value for isPartOf value \ THEN the dataset detail page should indicate it is part of a Collection.
[x] GIVEN a dataset with a value for isPartOf value \ AND the parent dataset is active THEN the dataset detail page should have a link to the parent dataset.
[x] GIVEN a dataset with a value for isPartOf value \ AND the parent dataset is not active (deleted or does not exist) THEN the dataset detail page should have a link to its Collection page

Background

The UI should be similar to the Collection in the existing catalog.data.gov. The difference is that there is no additional CKAN field in the extras collection_metadata=true to indicate a parent(collection) dataset. The collection relations are built by querying SOLR index with harvest_source_id and isPartOf values.

Security Considerations (required)

[Any security concerns that might be implicated in the change. "None" is OK, just be explicit here!]

Sketch

[Notes or a checklist reflecting our understanding of the selected approach]

FuhuXia commented 3 weeks ago

Collection page has its own story.

FuhuXia commented 1 week ago

Most of work is done in PR https://github.com/GSA/ckanext-geodatagov/pull/282, keeping collection UI and user experience same as current catalog, but on the backend it is rewritten use solr query to tell parent-child relationship, since ckan is not aware if dataset is part of a collection.

Some side effects of ckan loosing control of collection logic are:

children datasets can exist without parent dataset.
parent dataset can lose all children dataset and become a regular dataset
a dataset can be a parent, and at the same time it is a child to another dataset, it means nested collection (grand children)
it does not prevent two datasets becoming parent-child to each other. we can add some UI element to indicate the troubled relationship once the datasets are visited.

TODO add collection icon to parent datasets on the dataset listing page.

GSA / data.gov