Class 'Islandora\Tuque\Guzzle\Client' not found during migrations

[mjordan]

I'm running migrate_7x_claw on a clean claw-playbook. composer install ran without any issues, and www/html/drupal/web/modules/contrib/migrate_7x_claw/vendor/guzzlehttp exists.

The "Basic Image Objects" migration succeeds, but while performing the "Basic Image Objects OBJ" migration, I get the following error:

Error: Class 'Islandora\Tuque\Guzzle\Client' not found in Drupal\migrate_7x_claw\Plugin\migrate_plus\data_parser\TuqueDatastream::initializeApi() (line 144 of /var/www/html/drupal/web/modules/contrib/migrate_7x_claw/src/Plugin/migrate_plus/data_parser/TuqueDatastream.php) #0 /var/www/html/drupal/web/modules/contrib/migrate_7x_claw/src/Plugin/migrate_plus/data_parser/TuqueDatastream.php(127): Drupal\migrate_7x_claw\Plugin\migrate_plus\data_parser\TuqueDatastream::initializeApi('http://192.168....', 'fedoraAdmin', 'fedoraAdmin') #1 /var/www/html/drupal/web/modules/contrib/migrate_7x_claw/src/Plugin/migrate_plus/data_parser/TuqueDatastream.php(110): Drupal\migrate_7x_claw\Plugin\migrate_plus\data_parser\TuqueDatastream::initializeConnection('http://192.168....', 'fedoraAdmin', 'fedoraAdmin') #2 /var/www/html/drupal/web/modules/contrib/migrate_7x_claw/src/Plugin/migrate_plus/data_parser/TuqueDatastream.php(163): Drupal\migrate_7x_claw\Plugin\migrate_plus\data_parser\TuqueDatastream->getConnection() #3 /var/www/html/drupal/web/modules/contrib/migrate_plus/src/DataParserPluginBase.php(164): Drupal\migrate_7x_claw\Plugin\migrate_plus\data_parser\TuqueDatastream->openSourceUrl('http://192.168....') #4 /var/www/html/drupal/web/modules/contrib/migrate_7x_claw/src/Plugin/migrate_plus/data_parser/TuqueDatastream.php(233): Drupal\migrate_plus\DataParserPluginBase->nextSource() #5 /var/www/html/drupal/web/modules/contrib/migrate_plus/src/DataParserPluginBase.php(103): Drupal\migrate_7x_claw\Plugin\migrate_plus\data_parser\TuqueDatastream->nextSource() #6 /var/www/html/drupal/web/modules/contrib/migrate_plus/src/DataParserPluginBase.php(94): Drupal\migrate_plus\DataParserPluginBase->next() #7 /var/www/html/drupal/web/core/modules/migrate/src/Plugin/migrate/source/SourcePluginBase.php(328): Drupal\migrate_plus\DataParserPluginBase->rewind() #8 /var/www/html/drupal/web/core/modules/migrate/src/MigrateExecutable.php(188): Drupal\migrate\Plugin\migrate\source\SourcePluginBase->rewind() #9 /var/www/html/drupal/web/modules/contrib/migrate_tools/src/MigrateBatchExecutable.php(206): Drupal\migrate\MigrateExecutable->import() #10 /var/www/html/drupal/web/core/includes/batch.inc(294): Drupal\migrate_tools\MigrateBatchExecutable::batchProcessImport('islandora_basic...', Array, Array) #11 /var/www/html/drupal/web/core/includes/batch.inc(137): _batch_process() #12 /var/www/html/drupal/web/core/includes/batch.inc(93): _batch_do() #13 /var/www/html/drupal/web/core/modules/system/src/Controller/BatchController.php(55): _batch_page(Object(Symfony\Component\HttpFoundation\Request)) #14 [internal function]: Drupal\system\Controller\BatchController->batchPage(Object(Symfony\Component\HttpFoundation\Request)) #15 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array) #16 /var/www/html/drupal/web/core/lib/Drupal/Core/Render/Renderer.php(582): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #17 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure)) #18 /var/www/html/drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) #19 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(151): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #20 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(68): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1) #21 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/Session.php(57): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #22 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(47): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #23 /var/www/html/drupal/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(99): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #24 /var/www/html/drupal/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(78): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true) #25 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(47): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #26 /var/www/html/drupal/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(52): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #27 /var/www/html/drupal/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #28 /var/www/html/drupal/web/core/lib/Drupal/Core/DrupalKernel.php(665): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #29 /var/www/html/drupal/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request)) #30 {main}.

Looking at the code, I can confirm that migrate_7x_claw/src/Plugin/migrate_plus/data_parser/TuqueDatastream.php has a use Islandora\Tuque\Guzzle\Client; directive, which appears to match the namespace Islandora\Tuque\Guzzle; namespace used in migrate_7x_claw/vendor/jonathangreen/tuque/src/Guzzle/Client.php.

@whikloj @jonathangreen any suggestions? I also get a similar Guzzle client missing error when I execute the "Basic Image Objects OBJ Media" migration.

[DiegoPino]

Should not vendor/guzzlehttp be in then main Drupal Project Vendor instead of the particular module vendor? Drupal's symfony autoload is not able (nor any one i know of) to load from deep nested folders, since PSR-4 defines a folder based class loader

[mjordan]

@DiegoPino Guzzle is there: /var/www/html/drupal/vendor/guzzlehttp, with a namespace of GuzzleHttp in Client.php.

[mjordan]

Replacing use Islandora\Tuque\Guzzle\Client; with use GuzzleHttp\Client; in TuqueDatastream.php seems to move past the original error above, but now I get:

Error: Class 'Islandora\Tuque\Api\FedoraApi' not found in Drupal\migrate_7x_claw\Plugin\migrate_plus\data_parser\TuqueDatastream::initializeApi() (line 149 of /var/www/html/drupal/web/modules/contrib/migrate_7x_claw/src/Plugin/migrate_plus/data_parser/TuqueDatastream.php)

[...]

[DiegoPino]

And what about Islandora\Tuque\ ? That comes from

"jonathangreen/tuque": "dev-master" here https://github.com/Islandora-Devops/migrate_7x_claw/blob/master/composer.json#L20

So what could be confusing is that this particular composer.json is just a reference and only kicks in when deployed using the Drupal project way, if not, means git cloning this project, you need to do add those dependencies to main drupal manually

go to /var/www/html/drupal/ and do that composer require manually there

[mjordan]

@DiegoPino thanks, running composer require jonathangreen/tuque dev-master in the drupal base directory has removed the class not found errors.

So I guess this means that the install docs for the migrations should indicate that you run this command and not composer install in modules/contrib/migrate_7x_claw.

[mjordan]

Discussed on the 2018-10-10 CLAW call. @dannylamb will add migrate_7x_claw to Packagist, which will let us run composer update in the Drupal base directory to install dependencies sitewide. I will test and report back.

[DiegoPino]

@mjordan i have a question that is neither an issue or a task, more like a curious though. About migration, and you are the man for this. What is the use case, expected use, or need of moving fedora specific data, without transformation to Drupal? In other words, what do you all have planned on doing with-it -integrating-how with islandora 8? Only if you have time.

[mjordan]

@DiegoPino thanks for the complement but I am not the main person for migrations (well, maybe I currently am for the AUDIT datastream), but in my opinion the general motivation is to ensure that all 7.x datastreams of potential value are migrated, not just the image, video, etc. payloads and accompanying descriptive metadata. The only other common datastreams would be RELS* and and policy datastreams, and many sites have custom datastreams. (Of course some sites will not care about many of these datastreams so we should make migrating specific datastreams optional as long as their absence doesn't break core functionality.)

I'm most interested in the AUDIT datastream because it contains a record of all fixity check transactions (it also contains Fedora-generated addition of datastream transactions) generated by 7.x Checksum Checker. Once those fixity check transactions are migrated to CLAW, Riprap will migrate them (plugin, working test code) to its persistence layer so that we have the entire history of fixity checks since the repository started recording them for a given resource using Checksum Checker. That history is part of the "chain of preservation" that some sites will want for their migrated assets.

What are your views on what we should be migrating? I wonder if this discussion deserves its own issue.

[DiegoPino]

Thansk for this @mjordan. My thoughts are on the poor librarian (guessing, could be an intern or a CS person like me) that will be doing this (who loves to migrate?) and then end with data that could need to be transformed again to have use and could be even more complicated to do so once in Drupal. Eg. with Audits. Same goes for RELS-EXT. Many internal references are regarding IDs and resources that won't exist anymore in this new context or will live somewhere esle. So wondering if, as a side task, discussion could happen about what of the migrated data is useful immediately inside Drupal and which requires post processing and mashing-smashing to be Drupal aware? Or maybe all this could be deposited as a Bag somewhere to be never be retrieved until someone, in 2022, feels Fedora3 was the thing and the eternal loop starts again! I like this theoretical discussions, better to prevent than to heal is my 2018 moto.

[DiegoPino]

As an additional thought (this whole conversation will get lost because its in the wrong issue, but whatever), maybe, for those more preservation aware, Islandora 8 could simply handle a Bagit content type? You can bag-it audit too right? All in a single package? i need more coffee

[mjordan]

Yes, we totally should be discussing this. I am a big fan of Bagit for sure. But what do you do with the bags? I guess my assumption is that the Fedora repository is where most of the durable content lives, so migrating and storing each 7.x datstream of potential interest as binary resources in your repo linked to the object is one choice, as is bagging the whole 7.x object up and storing it as a single binary resource in the repo. Or storing it somewhere else. Doing the latter two things is not precluded by the current (and very early baby steps) approach of migrating all the datastreams. Providing the option to not migrate specific datastreams so that repo admins/migration teams can have some control seems like a reasonable approach to me. Then they can bag up what they are interested in from the source repo and do with it whatever their institutional preservation policy requires.

[mjordan]

@DiegoPino it's probably time we started a new issue....

[dannylamb]

@mjordan https://github.com/Islandora-Devops/migrate_7x_claw is now available. And even auto-updated without Github services. That makes two we've cleaned out this week :dagger:

Now you should be able to composer require islandora/migrate_7x_claw from /var/www/html/drupal and everything will sort itself out.

[mjordan]

Cool, will test after lunch.

[dannylamb]

@mjordan @DiegoPino The thought at the time was to bring everything over as-is, erring on the side of bringing over too much rather than leaving something behind. But yes, ideally, a lot of this will get processed and turned into fields, and we haven't pinned down the full scope of that yet. OCR, FITS, RELS-EXT, RELS-INT, MODS (or whatever other xml standard you're using for description) can all get values xpath'd out in the migration yml. I'd say a new ticket is definitely warranted.

[mjordan]

Works great, thanks everyone. I'll update the migration modules' README and open a PR tonight or tomorrow.