OAI-PMH migrations do not work

steinmb commented 5 years ago

Currently when you go to the migrate page you get a WSOD - http://cellproject.net/admin/content/migrate

resulted in a500 Service unavailable`

ELMCIP D7 endpoint uses: https://github.com/whiskyechobravo/views_oai_pmh

Suggested fix

Re-implement data import in Drupal 9. This avoid us having to re-implement this in both D7 and D9.

Drupal 9 endpoint support: https://www.drupal.org/project/rest_oai_pmh
Drupal 9 endpoint metadata: https://www.drupal.org/project/metatag

Overview

Dependencies on https://www.drupal.org/project/migrate_oaipmh

 Group: ADEL            Total  Imported  Unprocessed  Status  Last imported
 ADELCellNode           N/A    799       -799         Idle    2018-05-29 16:41:13

 Group: Ciberia         Total  Imported  Unprocessed  Status  Last imported
 CiberiaCellNode        N/A    54        -58          Idle    2018-05-04 16:17:52

 Group: ELD             Total  Imported  Unprocessed  Status  Last imported
 ELDCellNode            N/A    483       -483         Idle    2018-05-04 15:04:48

 Group: ELMCIP          Total  Imported  Unprocessed  Status  Last imported
 ELMCIPCellNode         N/A    2844      -2844        Idle    2018-05-09 06:37:35

 Group: NT2CELLOAIPMH   Total  Imported  Unprocessed  Status  Last imported
 NT2CELLOAIPMHCellNode  N/A    180       -180         Idle    2018-05-04 14:49:46

 Group: POEX            Total  Imported  Unprocessed  Status  Last imported
 POEXCellNode           N/A    5         -5           Idle    2018-05-04 14:52:23

Request URL

http://nt2.uqam.ca/en/oai-pmh?verb=ListRecords&metadataPrefix=mods
http://elmcip.net/data/work/oai-pmh?verb=ListRecords&metadataPrefix=mods
http://directory.eliterature.org/cell-oai-pmh?verb=ListRecords
https://adel.uni-siegen.de/cell-oai-pmh?verb=ListRecords
cURL error 6: Could not resolve host: beta.po-ex.net (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)

Field mapping

      'identifier' => 'header/identifier',
      'mods/recordInfo/recordContentSource' => 'mods/recordInfo/recordContentSource',
      'mods/titleInfo/title' => 'mods/titleInfo/title',
      'mods/originInfo/dateIssued' => 'mods/originInfo/dateIssued',
      'mods/language/languageTerm' => 'mods/language/languageTerm',
      'mods/location/url' => 'mods/location/url',
      'mods/identifier' => 'mods/identifier',
      'mods/extension/recordInfo/name/namePart' => 'mods/extension/recordInfo/name/namePart',
      'mods/physicalDescription/form/technology' => 'mods/physicalDescription/form/technology',
      'mods/physicalDescription/form/publication-type' => t('mods/physicalDescription/form/publication-type (with authority)'),
      'mods/physicalDescription/form/publication-type-complementary' => t('mods/physicalDescription/form/publication-type (without authority)'),
      'mods/genre/procedural-modality' => t('mods/genre/procedural-modality (with authority)'),
      'mods/genre/procedural-modality-complementary' => t('mods/genre/procedural-modality (without authority)'),
      'mods/physicalDescription/form/mechanism' => t('mods/physicalDescription/form/mechanism (with authority)'),
      'mods/physicalDescription/form/mechanism-complementary' => t('mods/physicalDescription/form/mechanism (without authority)'),
      'mods/physicalDescription/form/format' => t('mods/physicalDescription/form/format (with authority)'),
      'mods/physicalDescription/form/format-complementary' => t('mods/physicalDescription/form/format (without authority)'),
      'mods/abstract' => 'mods/abstract',
      'mods/genre/literary-quality' => 'mods/genre/literary-quality',
      'mods/recordInfo/recordCreationDate' => 'mods/recordInfo/recordCreationDate',
      'mods/recordInfo/recordChangeDate' => 'mods/recordInfo/recordChangeDate',
      'mods/recordInfo/languageOfCataloging/languageTerm' => 'mods/recordInfo/languageOfCataloging/languageTerm',

How to test

Migration should work as expected and running directly in CLI should throw no unexpected errors or warnings. drush migrate:import <group>

steinmb commented 3 years ago

Retested. It now time out after 30 sec.

steinmb commented 2 years ago

Going to be addressed in D9. Blocked on upgrade

steinmb commented 1 year ago

Drupal 9 upgrade is done and we can start working on getting this working again.

smbjorklund / cellproject