cessda / cessda.cdc.versions

Issue track and wiki for the CESSDA Data Catalogue
https://datacatalogue.cessda.eu/
Apache License 2.0
0 stars 0 forks source link

Remove DANS endpoint from production #662

Closed john-shepherdson closed 6 months ago

john-shepherdson commented 6 months ago

DANS endpoint configuration was withheld from 3.4.0 release, but it appears that it found its way into the 3.5.0 release. DANS records in both Dutch and English can be seen in the English language selection. The SO would like the DANS endpoint and associated records to be removed from production and excluded from future releases, until further notice.

matthew-morris-cessda commented 6 months ago

DANS removed from production

john-shepherdson commented 5 months ago

See also #605

RicardaBraukmann commented 5 months ago

Hi everyone, I would like to solve this so that our metadata can be included in the CDC. Asking my colleagues they told me that we have to sets (an EN and a Dutch one) and I believe these were previously used to get our metadata into the CDC. https://ssh.datastations.nl/oai?verb=ListRecords&metadataPrefix=oai_dc&set=CESSDA-EN https://ssh.datastations.nl/oai?verb=ListRecords&metadataPrefix=oai_dc&set=CESSDA-NL

Do I understand it correctly that having these sets is not sufficient? Our metadata actually does include a "language of metadata" attribute which is even now made mandatory, but it might not be included in all of the metadata exports Dataverse provides.

If these sets are insufficient, could you give us more details about what would be required and what is harvested from Dataverse? Since other SPs using Dataverse are included in the CDC I hope we could adjust our exports to comply with your requirements.

Many thanks. Ricarda

john-shepherdson commented 5 months ago

We could use the sets as above, but would have to treat them as 2 different endpoints with different names and different default languages. Might be confusing for the users to see publishes called (for example) 'DANS-KNAW (English)' and 'DANS-KNAW (Dutch)' - also the names would not comply with the Publisher names CV (https://vocabularies.cessda.eu/vocabulary/CdcPublisherNames?lang=en)

RicardaBraukmann commented 5 months ago

Thanks @john-shepherdson for looking into it. For us I would prefer to be included in some way so as soon as possible so if what you say is possible that would be great. Alternatively, you could also for now harvest the English records only as those will be most relevant for CDC users I believe and that set is also our bigger set from the two.

Of course we want to be included full as soon as possible so it would be great if we can discuss how that can be achieved.

Can you specify what we need to do in order to be harvested through our regular endpoint?

We have language of metadata information in our metadata in a custom block so the information is available for most datasets. I am not sure how you harvest the Dataverse instances (i.e. what metadata schema do you use), and what adjustments we would need to make to comply with the requirements? I am happy to connect you with our technical team as well as they know better how things are currently implemented.

matthew-morris-cessda commented 5 months ago

Discussion about re-adding DANS metadata to CDC should be discussed in a separate issue

john-shepherdson commented 5 months ago

See #667 re adding DANS endpoint