Export of data through OAI-PMH endpoints - Githubissues

sci2zero / TeslaRIS-backend

GNU General Public License v3.0

0 stars 0 forks source link

Export of data through OAI-PMH endpoints #22

Open chenejac opened 1 year ago

chenejac commented 1 year ago

Is your feature request related to a problem? Please describe. Export of data through OAI-PMH protocol is needed for joining OpenAIRE, ENauka and other aggregation service providers

Describe the solution you'd like We need easy customization of numerous endpoints

which data/records are available through endpoint
which metadata formats are supported
which sets are supported

Moreover, we need an easy way to define different crosswalks (mappings) between our internal model and target format (Dublin Core, Dspace internal model, OpenAIRE guideline for CRIS managers, ETD-MS format, etc.)

Additional context

There are some libraries which might be used for implementation of OAI-PMH protocol, the OAICat is one example (used in CRIS UNS)
There are online validators for OAI-PMH endpoints - https://proref.rcub.bg.ac.rs/OAIValidator/index.xhtml, https://provide.openaire.eu/about, etc.
There are also mapping languages and engines for execution of those mappings (RDF mapping language, X3ML tool set, etc.)

chenejac commented 1 month ago

The OAI-PMH specification - https://www.openarchives.org/OAI/openarchivesprotocol.html
Dublin Core specification - https://www.openarchives.org/OAI/2.0/oai_dc.xsd
ETD MS - http://cris.uns.ac.rs/etdms/1.1/etdms11.xsd
dim (DSpace internal model) - http://www.dspace.org/schema/dim.xsd (I can't find XSD file, maybe this can help - https://github.com/DSpace/DSpace/blob/main/dspace/config/crosswalks/oai/metadataFormats/dim.xsl, here is mapping in CRIS UNS - https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/AbstractDIMXMLSerializer.java, https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/NaRDuSDIMXMLSerializer.java)
oai_cerif_openaire - https://www.openaire.eu/schema/cris/current/openaire-cerif-profile.xsd (mapper in CRIS UNS https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/OpenAIRECRISXMLSerializer.java)

The existing OAI-PMH endpoints in CRIS UNS:

DOI Serbia https://doiserbia.nb.rs/phd/university.aspx?universityid=SR0223-01
- Identify https://www.cris.uns.ac.rs/OAIHandlerDOISerbia?verb=Identify
- Sets https://www.cris.uns.ac.rs/OAIHandlerDOISerbia?verb=ListSets
  - theses
- Formats https://www.cris.uns.ac.rs/OAIHandlerDOISerbia?verb=ListMetadataFormats
  - oai_dc (Dublin Core) https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/AbstractDublinCoreXMLSerializer.java, and https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/DOISerbiaDublinCoreXMLSerializer.java
- Records
  - DC example https://www.cris.uns.ac.rs/OAIHandlerDOISerbia?verb=ListRecords&metadataPrefix=oai_dc&from=2024-03-21&until=2024-03-27&set=theses
  - Identifiers
    - an example https://www.cris.uns.ac.rs/OAIHandlerDOISerbia? verb=ListIdentifiers&metadataPrefix=oai_dc&from=2024-03-21&until=2024-03-27&set=theses
  - Record
    - DC example https://www.cris.uns.ac.rs/OAIHandlerDOISerbia?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai:CRISUNS:(BISIS)100637
DOI Serbia PA https://oatd.org/
- The same as DOI Serbia, but for University Business Academy (the one above is for University of Novi Sad) , the endpoint is https://www.cris.uns.ac.rs/OAIHandlerOATDPA
DARTEurope https://www.dart-europe.org/browse-results.php?institution=578
DARTEurope PA https://oatd.org/
- The same as DARTEurope, but for University Business Academy (the one above is for University of Novi Sad) , the endpoint is https://www.cris.uns.ac.rs/OAIHandlerDARTEuropePA
OpenAIRE (disertacije) - https://explore.openaire.eu/search/dataprovider?datasourceId=opendoar____::ae1d2c2d957a01dcb3f3b39685cdb4fa
-OpenAIRE PA https://oatd.org/
- The same as OpenAIRE, but for University Business Academy (the one above is for University of Novi Sad) , the endpoint is https://www.cris.uns.ac.rs/OAIHandlerOpenAIREPA
NDLTD http://search.ndltd.org/search.php?q=publisher%3A+%22Novi+Sad%22
- Identify https://www.cris.uns.ac.rs/OAIHandlerNDLTD?verb=Identify
- Sets https://www.cris.uns.ac.rs/OAIHandlerNDLTD?verb=ListSets
  - theses
- Formats https://www.cris.uns.ac.rs/OAIHandlerNDLTD?verb=ListMetadataFormats
  - oai_dc (Dublin Core) https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/AbstractDublinCoreXMLSerializer.java, and https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/DOISerbiaDublinCoreXMLSerializer.java
  - oai_etdms (ETD MS) https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/AbstractETDMSXMLSerializer.java, and https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/NDLTDETDMSXMLSerializer.java
- Records
- NDLTD PA http://search.ndltd.org/search.php
- The same as NDLTD, but for University Business Academy (the one above is for University of Novi Sad) , the endpoint is https://www.cris.uns.ac.rs/OAIHandlerNDLTDPA
OATD https://oatd.org/
- The same as NDLTD, the endpoint is https://www.cris.uns.ac.rs/OAIHandlerOATD
OATD PA https://oatd.org/
- The same as OATD, but for University Business Academy (the one above is for University of Novi Sad) , the endpoint is https://www.cris.uns.ac.rs/OAIHandlerOATDPA
NaRDuS https://nardus.mpn.gov.rs/discover?filtertype=university&filter_relational_operator=equals&filter=%D0%A3%D0%BD%D0%B8%D0%B2%D0%B5%D1%80%D0%B7%D0%B8%D1%82%D0%B5%D1%82+%D1%83+%D0%9D%D0%BE%D0%B2%D0%BE%D0%BC+%D0%A1%D0%B0%D0%B4%D1%83
OpenAIRE (CRIS Faculty of Science, University of Novi Sad results) - https://explore.openaire.eu/search/dataprovider?datasourceId=eurocrisdris::30b9090b9374534690d03e6bb06f75be
- Identify https://www.cris.uns.ac.rs/OAIHandlerOpenAIRECRIS?verb=Identify
- Sets https://www.cris.uns.ac.rs/OAIHandlerOpenAIRECRIS?verb=ListSets
  - openaire_cris_equipments
  - openaire_cris_events
  - openaire_cris_funding
  - openaire_cris_orgunits
  - openaire_cris_patents
  - openaire_cris_persons
  - openaire_cris_products
  - openaire_cris_projects
  - openaire_cris_publications
- Formats https://www.cris.uns.ac.rs/OAIHandlerOpenAIRECRIS?verb=ListMetadataFormats
  - oai_dc (Dublin Core) https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/AbstractDublinCoreXMLSerializer.java, and https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/DOISerbiaDublinCoreXMLSerializer.java
  - oai_cerif_openaire - https://www.openaire.eu/schema/cris/current/openaire-cerif-profile.xsd, https://github.com/chenejac/cris/blob/master/src/rs/ac/uns/ftn/informatika/bibliography/interoperability/serializers/OpenAIRECRISXMLSerializer.java
- Records
  - oai_cerif_openaire publications example https://www.cris.uns.ac.rs/OAIHandlerOpenAIRECRIS?verb=ListRecords&metadataPrefix=oai_cerif_openaire&from=2024-03-21&until=2024-03-27&set=openaire_cris_publications
  - Record
    - publication example https://www.cris.uns.ac.rs/OAIHandlerOpenAIRECRIS?verb=GetRecord&metadataPrefix=oai_cerif_openaire&identifier=oai:CRIS.UNS:Publications/(BISIS)101741
    - person example https://www.cris.uns.ac.rs/OAIHandlerOpenAIRECRIS?verb=GetRecord&metadataPrefix=oai_cerif_openaire&identifier=oai:CRIS.UNS:Persons/(BISIS)936
ProQuest https://www.proquest.com/
- The same as DOI Serbia, the endpoint is https://www.cris.uns.ac.rs/OAIHandlerProQuest
BEOPEN https://open.uns.ac.rs/
- Identify https://www.cris.uns.ac.rs/OAIHandlerBEOPEN?verb=Identify
- Sets https://www.cris.uns.ac.rs/OAIHandlerBEOPEN?verb=ListSets
  - theses (PhD)
  - publications (CRIS)
- Formats https://www.cris.uns.ac.rs/OAIHandlerNaRDuS?verb=ListMetadataFormats
  - dim (DSpace internal model)
  - oai_dc (Dublin Core)
- Records
  - dim publications example https://www.cris.uns.ac.rs/OAIHandlerBEOPEN?verb=ListRecords&metadataPrefix=dim&from=2024-03-21&until=2024-03-27&set=publications
  - dim theses example https://www.cris.uns.ac.rs/OAIHandlerBEOPEN?verb=ListRecords&metadataPrefix=dim&from=2024-03-21&until=2024-03-27&set=theses
  - Identifiers
    - an example dim theses - https://www.cris.uns.ac.rs/OAIHandlerBEOPEN?verb=ListIdentifiers&metadataPrefix=dim&from=2024-03-21&until=2024-03-27&set=theses
    - an example dim publications - https://www.cris.uns.ac.rs/OAIHandlerBEOPEN?verb=ListIdentifiers&metadataPrefix=dim&from=2024-03-21&until=2024-03-27&set=publications
  - Record
    - dim thesis example https://www.cris.uns.ac.rs/OAIHandlerBEOPEN?verb=GetRecord&metadataPrefix=dim&identifier=oai:CRIS.UNS:Publications/(BISIS)110896
    - dim publication example https://www.cris.uns.ac.rs/OAIHandlerBEOPEN?verb=GetRecord&metadataPrefix=dim&identifier=oai:CRIS.UNS:Publications/(BISIS)101741
BEOPEN ILFE https://open.uns.ac.rs/
- The same as BEOPEN, but for https://ilfe.org/en (the one above is for Faculty of Sciences, University of Novi Sad) , the endpoint is https://www.cris.uns.ac.rs/OAIHandlerBEOPENILFE
eNauka https://enauka.gov.rs/cris/ou/ou00225
- Identify https://www.cris.uns.ac.rs/OAIHandlerENauka?verb=Identify
- Sets https://www.cris.uns.ac.rs/OAIHandlerENauka?verb=ListSets
  - openaire_cris_equipments
  - openaire_cris_events
  - openaire_cris_funding
  - openaire_cris_orgunits
  - openaire_cris_projects
  - openaire_cris_persons
  - openaire_cris_products
  - openaire_cris_patents
  - openaire_cris_publications (only this one and previous two have been used by enauka for my best knowledge, dim format is used)
- Formats https://www.cris.uns.ac.rs/OAIHandlerENauka?verb=ListMetadataFormats
  - dim (DSpace internal model)
  - oai_dc (Dublin Core)
- Records
  - dim publications example https://www.cris.uns.ac.rs/OAIHandlerENauka?verb=ListRecords&metadataPrefix=dim&from=2024-03-21&until=2024-03-27&set=openaire_cris_publications
  - Identifiers
    - an example dim publications - https://www.cris.uns.ac.rs/OAIHandlerENauka?verb=ListIdentifiers&metadataPrefix=dim&from=2024-03-21&until=2024-03-27&set=openaire_cris_publications
  - Record
    - dim publication example https://www.cris.uns.ac.rs/OAIHandlerENauka?verb=GetRecord&metadataPrefix=dim&identifier=oai:CRIS.UNS:Publications/(BISIS)101741
eNauka ILFE https://enauka.gov.rs/cris/ou/ou00318
- The same as eNauka, but for https://ilfe.org/en (the one above is for Faculty of Sciences, University of Novi Sad) , the endpoint is https://www.cris.uns.ac.rs/OAIHandlerENaukaILFE
eNauka UPA Farmacija https://enauka.gov.rs/cris/ou/ou00351
- The same as eNauka, but for https://www.privrednaakademija.edu.rs/fakulteti/farmaceutski-fakultet-novi-sad.html (the two above is for Faculty of Sciences, University of Novi Sad) , the endpoint is https://www.cris.uns.ac.rs/OAIHandlerENaukaUPAFarmacija