az-digital / az_quickstart

UArizona's web content management system built with Drupal 10.
https://quickstart.arizona.edu
GNU General Public License v2.0
30 stars 20 forks source link

Migration fails on stub creation when source site has deleted entity embedded in content. #1740

Closed trackleft closed 5 months ago

trackleft commented 2 years ago

Problem/Motivation

Would like migrations to run more smoothly

Describe the bug

Receive Call to a member function getSource() on null in Drupal\media\Entity\Media->getSource()

[error]  Error: Call to a member function getSource() on null in Drupal\media\Entity\Media->getSource() (line 137 of /app/web/core/modules/media/src/Entity/Media.php) #0 /app/web/core/modules/media/src/Entity/Media.php(371): Drupal\media\Entity\Media->getSource()
#1 /app/web/core/modules/media/src/MediaStorage.php(27): Drupal\media\Entity\Media->prepareSave()
#2 /app/web/core/lib/Drupal/Core/Entity/EntityBase.php(339): Drupal\media\MediaStorage->save(Object(Drupal\media\Entity\Media))
#3 /app/web/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php(247): Drupal\Core\Entity\EntityBase->save()
#4 /app/web/core/modules/migrate/src/Plugin/migrate/destination/EntityContentBase.php(186): Drupal\migrate\Plugin\migrate\destination\EntityContentBase->save(Object(Drupal\media\Entity\Media), Array)
#5 /app/web/core/modules/migrate/src/MigrateStub.php(118): Drupal\migrate\Plugin\migrate\destination\EntityContentBase->import(Object(Drupal\migrate\Row))
#6 /app/web/core/modules/migrate/src/MigrateStub.php(80): Drupal\migrate\MigrateStub->doCreateStub(Object(Drupal\migrate\Plugin\Migration), Array, Array)
#7 /app/web/profiles/custom/az_quickstart/modules/custom/az_migration/src/Plugin/migrate/process/EntityEmbedProcess.php(166): Drupal\migrate\MigrateStub->createStub('az_media', Array)
#8 /app/web/profiles/custom/az_quickstart/modules/custom/az_migration/src/Plugin/migrate/process/EntityEmbedProcess.php(254): Drupal\az_migration\Plugin\migrate\process\EntityEmbedProcess->updateEmbedTag('575', 'media', 'drupal-media', 'az_card_image', Object(DOMDocument), Object(DOMElement), 'az_media', 'media')
#9 /app/web/core/modules/migrate/src/MigrateExecutable.php(443): Drupal\az_migration\Plugin\migrate\process\EntityEmbedProcess->transform('<p>Lorem ipsum<...', Object(Drupal\migrate_tools\MigrateExecutable), Object(Drupal\migrate\Row), 'value')
#10 /app/web/core/modules/migrate/src/MigrateExecutable.php(392): Drupal\migrate\MigrateExecutable->processPipeline(Object(Drupal\migrate\Row), 'value', Array, '<p>Lorem ipsum<...')
#11 /app/web/core/modules/migrate/src/Plugin/migrate/process/SubProcess.php(207): Drupal\migrate\MigrateExecutable->processRow(Object(Drupal\migrate\Row), Array)
#12 /app/web/core/modules/migrate/src/MigrateExecutable.php(443): Drupal\migrate\Plugin\migrate\process\SubProcess->transform(Array, Object(Drupal\migrate_tools\MigrateExecutable), Object(Drupal\migrate\Row), 'field_az_text_a...')
#13 /app/web/core/modules/migrate/src/MigrateExecutable.php(221): Drupal\migrate\MigrateExecutable->processPipeline(Object(Drupal\migrate\Row), 'field_az_text_a...', Array, Array)
#14 /app/vendor/drush/drush/includes/drush.inc(122): Drupal\migrate\MigrateExecutable->import()
#15 /app/vendor/drush/drush/includes/drush.inc(113): drush_call_user_func_array(Array, Array)
#16 /app/web/modules/contrib/migrate_tools/src/Commands/MigrateToolsCommands.php(963): drush_op(Array)
#17 [internal function]: Drupal\migrate_tools\Commands\MigrateToolsCommands->executeMigration(Object(Drupal\migrate\Plugin\Migration), 'az_paragraph_he...', Array)
#18 /app/web/modules/contrib/migrate_tools/src/Commands/MigrateToolsCommands.php(417): array_walk(Array, Array, Array)
#19 [internal function]: Drupal\migrate_tools\Commands\MigrateToolsCommands->import('', Array)
#20 /app/vendor/consolidation/annotated-command/src/CommandProcessor.php(257): call_user_func_array(Array, Array)
#21 /app/vendor/consolidation/annotated-command/src/CommandProcessor.php(212): Consolidation\AnnotatedCommand\CommandProcessor->runCommandCallback(Array, Object(Consolidation\AnnotatedCommand\CommandData))
#22 /app/vendor/consolidation/annotated-command/src/CommandProcessor.php(176): Consolidation\AnnotatedCommand\CommandProcessor->validateRunAndAlter(Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
#23 /app/vendor/consolidation/annotated-command/src/AnnotatedCommand.php(350): Consolidation\AnnotatedCommand\CommandProcessor->process(Object(Symfony\Component\Console\Output\ConsoleOutput), Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
#24 /app/vendor/symfony/console/Command/Command.php(255): Consolidation\AnnotatedCommand\AnnotatedCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#25 /app/vendor/symfony/console/Application.php(1027): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#26 /app/vendor/symfony/console/Application.php(273): Symfony\Component\Console\Application->doRunCommand(Object(Consolidation\AnnotatedCommand\AnnotatedCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#27 /app/vendor/symfony/console/Application.php(149): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#28 /app/vendor/drush/drush/src/Runtime/Runtime.php(124): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#29 /app/vendor/drush/drush/src/Runtime/Runtime.php(51): Drush\Runtime\Runtime->doRun(Array, Object(Symfony\Component\Console\Output\ConsoleOutput))
#30 /app/vendor/drush/drush/drush.php(72): Drush\Runtime\Runtime->run(Array)
#31 /app/vendor/drush/drush/drush(4): require('/app/vendor/dru...')
#32 {main}. 

To Reproduce

Source site: Create source site with QS1 codebase Add media to be embedded Create page Add headed text paragraph to page Embed the media Save Go delete the media

Destination Site Set up site to be able to migration from the source site. Run the az_migration group

Proposed resolution

Found this issue on drupal.org about it. https://www.drupal.org/project/migrate_file_to_media/issues/3138268#comment-13669457

Possibly could be fixed by adding default values for bundle to the createStub call

https://github.com/az-digital/az_quickstart/blob/main/modules/custom/az_migration/src/Plugin/migrate/process/EntityEmbedProcess.php#L165-L167

https://api.druphelp.com/api/drupal/core%21modules%21migrate%21src%21MigrateStub.php/function/MigrateStub%3A%3AcreateStub/9

akslay commented 2 years ago

Would it be reasonable to maybe include a check during the migration script? As it's going through and working on the entity embeds, can we first include a check for something along the lines of ! if file exists on the source site, and if the file doesn't exist, then don't include the embed?

trackleft commented 2 years ago

Possibly add try catch to skip creating a stub. Also remove and replace drupal-entity element with <h1>entity no longer exists</h1>

Searchable using drupal search ^

joeparsons commented 2 years ago

Seems related: https://www.drupal.org/project/drupal/issues/3156730

trackleft commented 2 years ago

To bypass this if you do not need stubs you can change https://github.com/az-digital/az_quickstart/blob/main/modules/custom/az_migration/src/Plugin/migrate/process/EntityEmbedProcess.php#L164-L178 To

    if (empty($ids)) {
  //    $ids = $this->migrateStub->createStub($migration, [$id], $stub_defaults);
      $ids = "deleted_file";
    }
    // We eventually found our id, by lookup or stubbing it.
    if (!empty($ids)&&is_array($ids)) {
      $id = reset($ids);
      if (!empty($id)) {
        $eid = reset($id);
        $entity = $this->entityTypeManager->getStorage($storage)->load($eid);
        if ($entity) {
          $changed->setAttribute('data-entity-uuid', $entity->uuid());
        }
      }
    }
    else {
      $changed = $dom->createElement("h1", "deleted_entity");
    }
trackleft commented 2 years ago

New patch: Note: This patch will stop stub creation for embedded entities, so if you need that you need to use another approach.

        "patches": {
            "az-digital/az_quickstart" {
                "Migration fails on stub creation when source site has deleted entity embedded in content.(https://github.com/az-digital/az_quickstart/issues/1740)": "https://patch-diff.githubusercontent.com/raw/az-digital/az_quickstart/pull/1799.diff"
            }
        }

Remove this patch when done with your migration.

If you run into issues with this patch, please report them on this issue.

joeparsons commented 5 months ago

2024-05-15 meeting discussion

As far as we know this is no longer an issue. Issue can be re-opened if still affecting anyone.