Closed samhibberd closed 4 years ago
Appears resolved after reindexing the assets, but not sure what caused, be good to understand to avoid the site slowdown if it crops up again.
What version of the S3 plugin are you running?
We are running version 1.2.5 of craftcms/aws-s3
.
It looks like were a fair few recent updates to the aws/aws-sdk-php
package https://github.com/aws/aws-sdk-php/blob/master/CHANGELOG.md, not sure if there is anything in there.
Huh, alright, well let me know if it crops up again.
Similar situation for me:
2020-02-10 14:19:43 [-][1][9e0d1899ce9d3cb62e4db95eee647c7f][error][RuntimeException] RuntimeException: Instances of craft\awss3\S3Client cannot be serialized in /app/vendor/aws/aws-sdk-php/src/AwsClient.php:263
Stack trace:
#0 [internal function]: Aws\AwsClient->__sleep()
#1 /app/vendor/craftcms/cms/src/services/TemplateCaches.php(194): serialize(Object(craft\elements\db\EntryQuery))
#2 [internal function]: craft\services\TemplateCaches->includeElementQueryInTemplateCaches(Object(craft\events\CancelableEvent))
#3 /app/vendor/yiisoft/yii2/base/Event.php(312): call_user_func(Array, Object(craft\events\CancelableEvent))
...
The problem occurs when an EntryQuery
is being cached for an entry containing Aws\S3\S3Client
objects. Apparently this happens when S3 assets are not yet indexed? I found a similar issue reported in the laracasts.com forum. The following response in that thread seems to identify the issue:
Apparently, classes like S3Client cannot be serialized because it does not implement the JsonSerializable interface, that is necessary in order to serialize the object content in a database queue.
I can confirm that the problem is not present when template caching is disabled.
@markdrzy is this on frontend or backend?
@andris-sevcenko frontend, when template caching is enabled. It happens when rendering a page that contains an S3 asset that has not yet been indexed. If I have a chance today, I'll try to come up with steps to reproduce. Until then, this is what I think should trigger the issue:
Also, I can confirm that the error persists with Craft 3.4.5 and craftcms/aws-s3 1.2.7
I've started seeing this again recently @andris-sevcenko - about once a day....doesn't seem to be causing any major issues, and on re-loading the pages that triggered the error (logged in Rollbar) - it doesn't happen again.
My guess as to why I am seeing this more is because I am actively deploying small changes regularly at the moment, part of which involves nuking the template caches...but is this a bug, or something I should just mute in my tracking?
(I'm on the latest release versions of everything)
@bossanova808 similar stack trace as the original post? If so looks like it’s coming from the Debug module, which would only be loaded for admins that have it enabled in their user settings, so at least not something that effects normal visitor requests.
Here's a trace @brandonkelly
020-07-08 13:53:42 [-][-][cec8f545f6605f2d36bcae5ef64ff560][error][RuntimeException] RuntimeException: Instances of craft\awss3\S3Client cannot be serialized in /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/aws/aws-sdk-php/src/AwsClient.php:271
Stack trace:
#0 [internal function]: Aws\AwsClient->__sleep()
#1 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/services/TemplateCaches.php(177): serialize(Object(craft\commerce\elements\db\ProductQuery))
#2 [internal function]: craft\services\TemplateCaches->includeElementQueryInTemplateCaches(Object(craft\events\CancelableEvent))
#3 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/base/Event.php(312): call_user_func(Array, Object(craft\events\CancelableEvent))
#4 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/base/Component.php(636): yii\base\Event::trigger('craft\\elements\\...', 'afterPrepare', Object(craft\events\CancelableEvent))
#5 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/elements/db/ElementQuery.php(1922): yii\base\Component->trigger('afterPrepare', Object(craft\events\CancelableEvent))
#6 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/elements/db/ElementQuery.php(1413): craft\elements\db\ElementQuery->afterPrepare()
#7 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/db/QueryBuilder.php(227): craft\elements\db\ElementQuery->prepare(Object(craft\db\mysql\QueryBuilder))
#8 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/db/Query.php(157): yii\db\QueryBuilder->build(Object(craft\commerce\elements\db\ProductQuery))
#9 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/db/Query.php(318): yii\db\Query->createCommand(Object(craft\db\Connection))
#10 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/db/Query.php(202): yii\db\Query->column(NULL)
#11 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/elements/db/ElementQuery.php(1512): craft\db\Query->column(NULL)
#12 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/elements/db/ElementQuery.php(1552): craft\elements\db\ElementQuery->column(NULL)
#13 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/elements/db/ElementQuery.php(2512): craft\elements\db\ElementQuery->ids()
#14 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/elements/db/ElementQuery.php(1407): craft\elements\db\ElementQuery->_applySearchParam(Object(craft\db\Connection))
#15 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/db/QueryBuilder.php(227): craft\elements\db\ElementQuery->prepare(Object(craft\db\mysql\QueryBuilder))
#16 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/db/Query.php(157): yii\db\QueryBuilder->build(Object(craft\commerce\elements\db\ProductQuery))
#17 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/db/Query.php(248): yii\db\Query->createCommand(Object(craft\db\Connection))
#18 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/db/Query.php(152): yii\db\Query->all(NULL)
#19 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/elements/db/ElementQuery.php(1471): craft\db\Query->all(NULL)
#20 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/marionnewlevant/agnostic-fetch/src/helpers/AgnosticFetchHelper.php(21): craft\elements\db\ElementQuery->all()
#21 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/marionnewlevant/agnostic-fetch/src/twigextensions/AgnosticFetchTwigExtension.php(88): marionnewlevant\agnosticfetch\helpers\AgnosticFetchHelper::all(Object(craft\commerce\elements\db\ProductQuery))
#22 /srv/users/serverpilot/apps/imagescience/releases/dev/storage/runtime/compiled_templates/fc/fc8dcea214e3f482a85256ca6a7d46fcabfa00462b65906925bc5353fb831ef4.php(348): marionnewlevant\agnosticfetch\twigextensions\AgnosticFetchTwigExtension->marionnewlevant\agnosticfetch\twigextensions\{closure}(Object(craft\commerce\elements\db\ProductQuery))
#23 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(184): __TwigTemplate_1bbbf033ab7218ae8912230f02bff6e97b74f053262227d34d8e386258c6b997___2145468517->block_categoryList(Array, Array)
#24 /srv/users/serverpilot/apps/imagescience/releases/dev/storage/runtime/compiled_templates/46/46a91c2982aea6f21c54db7be9cc0bb60c5a69ecd03b64e76513a658dfe31c15.php(89): Twig\Template->displayBlock('categoryList', Array, Array)
#25 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(184): __TwigTemplate_2471139f0f278e0ee582c404888e902c0ca36ffdbf4129912c78bdbb66d63acb->block_structure(Array, Array)
#26 /srv/users/serverpilot/apps/imagescience/releases/dev/storage/runtime/compiled_templates/46/46a91c2982aea6f21c54db7be9cc0bb60c5a69ecd03b64e76513a658dfe31c15.php(47): Twig\Template->displayBlock('structure', Array, Array)
#27 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(407): __TwigTemplate_2471139f0f278e0ee582c404888e902c0ca36ffdbf4129912c78bdbb66d63acb->doDisplay(Array, Array)
#28 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(380): Twig\Template->displayWithErrorHandling(Array, Array)
#29 /srv/users/serverpilot/apps/imagescience/releases/dev/storage/runtime/compiled_templates/fc/fc8dcea214e3f482a85256ca6a7d46fcabfa00462b65906925bc5353fb831ef4.php(222): Twig\Template->display(Array, Array)
#30 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(407): __TwigTemplate_1bbbf033ab7218ae8912230f02bff6e97b74f053262227d34d8e386258c6b997___2145468517->doDisplay(Array, Array)
#31 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(380): Twig\Template->displayWithErrorHandling(Array, Array)
#32 /srv/users/serverpilot/apps/imagescience/releases/dev/storage/runtime/compiled_templates/fc/fc8dcea214e3f482a85256ca6a7d46fcabfa00462b65906925bc5353fb831ef4.php(160): Twig\Template->display(Array)
#33 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(184): __TwigTemplate_1bbbf033ab7218ae8912230f02bff6e97b74f053262227d34d8e386258c6b997->block_structure(Array, Array)
#34 /srv/users/serverpilot/apps/imagescience/releases/dev/storage/runtime/compiled_templates/0f/0fe1060329fca99b8a9c0e406a7ef3dd6e3ca28d09aa326679d02c4d8c908799.php(241): Twig\Template->displayBlock('structure', Array, Array)
#35 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(407): __TwigTemplate_e0f5d2e63426070931cfb1aaa44d07a18756859cc34b3bf0daa95b95e6769d46->doDisplay(Array, Array)
#36 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(380): Twig\Template->displayWithErrorHandling(Array, Array)
#37 /srv/users/serverpilot/apps/imagescience/releases/dev/storage/runtime/compiled_templates/fc/fc8dcea214e3f482a85256ca6a7d46fcabfa00462b65906925bc5353fb831ef4.php(145): Twig\Template->display(Array, Array)
#38 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(407): __TwigTemplate_1bbbf033ab7218ae8912230f02bff6e97b74f053262227d34d8e386258c6b997->doDisplay(Array, Array)
#39 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(380): Twig\Template->displayWithErrorHandling(Array, Array)
#40 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Template.php(392): Twig\Template->display(Array)
#41 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/TemplateWrapper.php(45): Twig\Template->render(Array, Array)
#42 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/twig/twig/src/Environment.php(318): Twig\TemplateWrapper->render(Array)
#43 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/web/View.php(375): Twig\Environment->render('_site/categoryI...', Array)
#44 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/web/View.php(436): craft\web\View->renderTemplate('_site/categoryI...', Array)
#45 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/web/Controller.php(235): craft\web\View->renderPageTemplate('_site/categoryI...', Array, 'site')
#46 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/controllers/TemplatesController.php(98): craft\web\Controller->renderTemplate('_site/categoryI...', Array)
#47 [internal function]: craft\controllers\TemplatesController->actionRender('_site/categoryI...', Array)
#48 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#49 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/base/Controller.php(157): yii\base\InlineAction->runWithParams(Array)
#50 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/web/Controller.php(178): yii\base\Controller->runAction('render', Array)
#51 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/base/Module.php(528): craft\web\Controller->runAction('render', Array)
#52 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/web/Application.php(291): yii\base\Module->runAction('templates/rende...', Array)
#53 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/web/Application.php(103): craft\web\Application->runAction('templates/rende...', Array)
#54 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/craftcms/cms/src/web/Application.php(276): yii\web\Application->handleRequest(Object(craft\web\Request))
#55 /srv/users/serverpilot/apps/imagescience/releases/dev/vendor/yiisoft/yii2/base/Application.php(386): craft\web\Application->handleRequest(Object(craft\web\Request))
#56 /srv/users/serverpilot/apps/imagescience/releases/dev/web/index.php(21): yii\base\Application->run()
#57 {main}
2020-07-08 13:53:40 [-][-][cec8f545f6605f2d36bcae5ef64ff560][info][application] $_GET = [
'p' => 'products/inkjet-papers/red-river'
]```
Ah, in your case it’s coming from the serialization of the product query when storing a template cache. We’ve completely rewritten how template caches work for Craft 3.5 in a way where this won’t be an issue anymore, and that’s just a couple weeks away ;)
That's do fine then!
We've noticed this error on a few of our sites, but under specific circumstances:
https://site.test/index.php?p=actions/debug/default/view&panel=request
){% do craft.app.elements.eagerLoadElements() %}
and then using the eager-loaded assets (e.g. asset.url
, asset.getUrl( transform )
)However, the original entry URL will load without error. I'm unsure how long this error has been happening as I've only recently had to enable the debug toolbar to investigate something and come across this.
Other notes:
enableTemplateCaching
, deleting storage/runtime
, clearing caches, and updating asset indexes doesn't make a differenceentry.assetField.one()
resolves the issue Example eager loading of entry that will error on debug URL:
{% do craft.app.elements.eagerLoadElements(
className( entry ),
[entry],
[
['testAsset', { kind: 'image' }],
['appointmentCollageSection.s:backgroundImage', { kind: 'image', withTransforms: ['collageHeroXl'] }]
]
) %}
After some digging, I think I may have found my issue!
It seems that omitting count
in either of these eager loading "formats" will cause the error:
{ path: 'quickLinks.link:backgroundImage', criteria: { kind: 'image', withTransforms: [...] } }
['quickLinks.link:backgroundImage', { kind: 'image', withTransforms: [...] }]
In other words, adding count
to the criteria seems to fix the error.
Looking at the craft files, my guess is that this line might be the culprit https://github.com/craftcms/cms/blob/4764694f6b974199b87b7e2a425a8d3da393187b/src/services/Elements.php#L1923
but will leave the rest to you.
Hey @hannahwoodward, the template cache system has been rewritten for Craft 3.5 and won’t ever cause these sorts of errors any more.
Hey @brandonkelly actually, my reported issue is happening on Craft 3.5.15.1, see above.
I don't think it's a template caching issue in this case - see https://github.com/craftcms/cms/issues/5395#issuecomment-723169564. So perhaps I should have created a new issue, sorry for tagging it on here. I can create a new issue if you'd prefer?
Yeah ok, this sounds like something new. Please create a new issue with full steps to reproduce.
I know it says we're not supposed to see this anymore, but I still do - again, in the debugger toolbar thing. Craft Pro 3.7.32. Again on product pages.
In dev mode, no template caching
'devMode' => (\craft\helpers\App::env('DEV_MODE') === 'true'),
'enableTemplateCaching' => (\craft\helpers\App::env('ENABLE_TEMPLATE_CACHING') === 'true'),
Trace:
RuntimeException: Instances of craft\awss3\S3Client cannot be serialized in /var/www/vhosts/c3-dev/vendor/aws/aws-sdk-php/src/AwsClient.php:275
Stack trace:
#0 [internal function]: Aws\AwsClient->__sleep()
#1 /var/www/vhosts/c3-dev/vendor/opis/closure/functions.php(20): serialize()
#2 /var/www/vhosts/c3-dev/vendor/yiisoft/yii2-debug/src/LogTarget.php(66): Opis\Closure\serialize()
#3 /var/www/vhosts/c3-dev/vendor/yiisoft/yii2-debug/src/LogTarget.php(136): yii\debug\LogTarget->export()
#4 /var/www/vhosts/c3-dev/vendor/yiisoft/yii2/log/Dispatcher.php(194): yii\debug\LogTarget->collect()
#5 /var/www/vhosts/c3-dev/vendor/yiisoft/yii2/log/Logger.php(226): yii\log\Dispatcher->dispatch()
#6 [internal function]: yii\log\Logger->flush()
#7 {main}
Any ideas on how to solve this?
@bossanova808 Are you setting any app component properties to closures in config/app.php
?
Not that I'm aware of...
<?php
return [
'modules' => [
'logger' => \modules\logger\Logger::class,
'site' => \modules\site\Site::class,
'dear' => \modules\dear\Dear::class,
'freshdesk' => \modules\freshdesk\Freshdesk::class,
'logic' => \modules\logic\Logic::class,
'messaging' => \modules\messaging\Messaging::class,
'shipper' => \modules\shipper\Shipper::class,
'bridge' => \modules\bridge\Bridge::class,
],
'bootstrap' => ['logger','dear','freshdesk','logic','messaging','shipper', 'bridge', 'site', ],
'components' => [
// Throw hard errors for deprecations on dev...nice! However, also throws them on console runs :(
// 'deprecator' => [
// 'throwExceptions' => YII_DEBUG,
// ],
'redis' => [
'class' => yii\redis\Connection::class,
'hostname' => getenv('REDIS_HOSTNAME'),
'port' => getenv('REDIS_PORT'),
'database' => getenv('REDIS_DEFAULT_DB'),
],
'cache' => [
'class' => yii\redis\Cache::class,
'redis' => [
'database' => getenv('REDIS_CRAFT_DB'),
],
'defaultDuration' => 86400,
'keyPrefix' => 'craft3_cache'
],
'session' => [
'class' => \yii\redis\Session::class,
'as session' => [
'class' => \craft\behaviors\SessionBehavior::class,
],
],
'queue' => [
'class' => craft\queue\Queue::class,
'ttr' => 60 * 60, // Minutes * seconds, i.e. 60 * 60 = one hour
],
// MYOB DB CONNECTIONS...
// https://craftcms.stackexchange.com/questions/28638/accessing-multiple-databases-in-craft
// Access via Craft::$app->myob_db_XX->createCommand(...);
'myob_db_20' => [
'class' => craft\db\Connection::class,
'driverName' => 'mysql',
'dsn' => 'mysql:host=localhost;dbname=MYOB20_PreFinal;port=3306;',
'username' => getenv('DB_USER'),
'password' => getenv('DB_PASSWORD'),
'charset' => 'utf8',
'tablePrefix' => '',
'enableSchemaCache' => !YII_DEBUG,
],
...a few more db connection indentical to the MYOB20 one follow....
],
];
From the stack trace, one of your Debug Toolbar panels are including a reference to an AwsClient object in their generated panel data for the request. Not clear which one though, or why. If you are able to step through this with Xdebug, try setting a breakpoint on Aws\AwsClient::__sleep()
(here). Then go back through the backtrace and see if you can tell what it’s trying to serialize in vendor/yiisoft/yii2-debug/src/LogTarget.php
.
Am just in the middle of migrating my dev setup to docker and setting up XDebug with that is part of the plan, so I'll get back to you on this once I get done with that - thanks Brandon!
I'm chiming in on this as well as I've see this occurring on certain requests but it's potential transient.
RuntimeException: Instances of craft\awss3\S3Client cannot be serialized
#23 /vendor/aws/aws-sdk-php/src/AwsClient.php(276): Aws\AwsClient::__sleep
#22 [internal](0): serialize
#21 /vendor/yiisoft/yii2/caching/Cache.php(249): yii\caching\Cache::set
#20 /vendor/yiisoft/yii2/caching/Cache.php(608): yii\caching\Cache::getOrSet
#19 /vendor/nystudio107/craft-seomatic/src/services/MetaContainers.php(271): nystudio107\seomatic\services\MetaContainers::parseGlobalVars
#18 /vendor/nystudio107/craft-seomatic/src/services/MetaContainers.php(249): nystudio107\seomatic\services\MetaContainers::includeMetaContainers
#17 /vendor/nystudio107/craft-seomatic/src/Seomatic.php(747): nystudio107\seomatic\Seomatic::nystudio107\seomatic\{closure}
#16 [internal](0): call_user_func
#15 /vendor/yiisoft/yii2/base/Event.php(312): yii\base\Event::trigger
#14 /vendor/yiisoft/yii2/base/Component.php(642): yii\base\Component::trigger
#13 /vendor/yiisoft/yii2/web/View.php(180): yii\web\View::endPage
#12 /vendor/craftcms/cms/src/web/View.php(1731): craft\web\View::endPage
#11 /vendor/craftcms/cms/src/web/View.php(462): craft\web\View::renderPageTemplate
#10 /vendor/craftcms/cms/src/web/Controller.php(201): craft\web\Controller::renderTemplate
#9 /vendor/craftcms/cms/src/controllers/TemplatesController.php(102): craft\controllers\TemplatesController::actionRender
#8 [internal](0): call_user_func_array
#7 /vendor/yiisoft/yii2/base/InlineAction.php(57): yii\base\InlineAction::runWithParams
#6 /vendor/yiisoft/yii2/base/Controller.php(178): yii\base\Controller::runAction
#5 /vendor/yiisoft/yii2/base/Module.php(552): yii\base\Module::runAction
#4 /vendor/craftcms/cms/src/web/Application.php(293): craft\web\Application::runAction
#3 /vendor/yiisoft/yii2/web/Application.php(103): yii\web\Application::handleRequest
#2 /vendor/craftcms/cms/src/web/Application.php(278): craft\web\Application::handleRequest
#1 /vendor/yiisoft/yii2/base/Application.php(384): yii\base\Application::run
#0 /index.php(12): null
Interestingly, SEOMatic is appearing in the stacktrace, so it's potentially something there.
Description
Seeing a strange request error in the debug toolbar and degraded performance on a site, nothing obvious has been changed, been working / testing the site both locally and on a staging environment with s3 handling all the assets.
Am i missing anything obvious?
Additional info