search

lsst-epo / investigations-api

Backend to the investigations 2.0 site
0 stars 0 forks source link

Review/Test webhooky syncing between Craft and Canto #82

Closed blnkt closed 5 months ago

blnkt commented 6 months ago

Including documenting test steps so that @blnkt may likewise replicate the testing.

khalwat commented 6 months ago

Do this before you test:

https://github.com/lsst-epo/canto-dam-assets/issues/8

...to re-save the entries that contain Canto DAM Assets fields so that the webhooks will work fully.

khalwat commented 6 months ago

The Webhook Controller Actions section of the README lists the 4 available webhooks.

These cover the use cases in the "Canto DAM Plugin GQL Query Use Cases" document:

ericdrosas87 commented 6 months ago

Per Slack conversation with @khalwat , matrix blocks do not seem to be converted over to from text type to jsonb and as such, I was seeing column-type errors in the queue logs when a webhook endpoint was hit. The webhook logic functions as expected, but the errors will persist until I manually convert over the text columns, which I have already done in dev, but not int nor prod yet. The risk for manually changing this column type is low as there is no data in this table and this matrix block type is currently unused per @blnkt .

ericdrosas87 commented 6 months ago

Error seen in queue logs for posterity:

2024-03-04 15:13:42 [queue.INFO] [yii\db\Connection::open] Opening DB connection: pgsql:host=10.109.176.18;dbname=investigations_feb_2024_v2;port=5432 {"memory":18229032} 
2024-03-04 15:13:42 [queue.INFO] [castiron\nextbuilds\NextBuilds::init] Next Builds plugin loaded {"memory":27897264} 
2024-03-04 15:13:42 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec]  [41980] t9n:["_canto-dam-assets","Updating Canto Asset id {id}",{"id":"f6e09958dl5hv5rn3ki1ecoa4s"}] (attempt: 1, pid: 59) - Started {"memory":29250744} 
2024-03-04 15:13:42 [queue.ERROR] [craft\queue\QueueLogBehavior::afterError]  [41980] t9n:["_canto-dam-assets","Updating Canto Asset id {id}",{"id":"f6e09958dl5hv5rn3ki1ecoa4s"}] (attempt: 1, pid: 59) - Error (time: 0.852s): SQLSTATE[42883]: Undefined function: 7 ERROR:  operator does not exist: text @> jsonb
LINE 3: ..."field_filterImage_image_cantoAssetData_pshetzuy" @> '[{"id"...
                                                             ^
HINT:  No operator matches the given name and argument types. You might need to add explicit type casts.
The SQL being executed was: SELECT "id", "field_filterImage_image_cantoAssetData_pshetzuy"
FROM "matrixcontent_filtersimages"
WHERE ("field_filterImage_image_pshetzuy"=0) AND (("field_filterImage_image_cantoAssetData_pshetzuy" @> '[{"id":"f6e09958dl5hv5rn3ki1ecoa4s"}]'::jsonb)) {"memory":34364520} 
2024-03-04 15:13:42 [queue.ERROR] [yii\db\Exception] PDOException: SQLSTATE[42883]: Undefined function: 7 ERROR:  operator does not exist: text @> jsonb
LINE 3: ..."field_filterImage_image_cantoAssetData_pshetzuy" @> '[{"id"...
                                                             ^
HINT:  No operator matches the given name and argument types. You might need to add explicit type casts. in /var/www/html/vendor/yiisoft/yii2/db/Command.php:1302
Stack trace:
#0 /var/www/html/vendor/yiisoft/yii2/db/Command.php(1302): PDOStatement->execute()
#1 /var/www/html/vendor/yiisoft/yii2/db/Command.php(1168): yii\db\Command->internalExecute()
#2 /var/www/html/vendor/yiisoft/yii2/db/Command.php(410): yii\db\Command->queryInternal()
#3 /var/www/html/vendor/yiisoft/yii2/db/Query.php(249): yii\db\Command->queryAll()
#4 /var/www/html/vendor/craftcms/cms/src/db/Query.php(247): yii\db\Query->all()
#5 /var/www/html/vendor/lsst-epo/canto-dam-assets/src/services/Assets.php(205): craft\db\Query->all()
#6 /var/www/html/vendor/lsst-epo/canto-dam-assets/src/services/Assets.php(154): lsst\cantodamassets\services\Assets->updateContent()
#7 /var/www/html/vendor/lsst-epo/canto-dam-assets/src/services/Assets.php(92): lsst\cantodamassets\services\Assets->updateBlockTypeContent()
#8 /var/www/html/vendor/lsst-epo/canto-dam-assets/src/services/Assets.php(42): lsst\cantodamassets\services\Assets->update()
#9 /var/www/html/vendor/lsst-epo/canto-dam-assets/src/jobs/UpdateByCantoId.php(28): lsst\cantodamassets\services\Assets->updateByCantoId()
#10 /var/www/html/vendor/yiisoft/yii2-queue/src/Queue.php(243): lsst\cantodamassets\jobs\UpdateByCantoId->execute()
#11 /var/www/html/vendor/yiisoft/yii2-queue/src/cli/Queue.php(162): yii\queue\Queue->handleMessage()
#12 /var/www/html/vendor/yiisoft/yii2-queue/src/cli/Command.php(146): yii\queue\cli\Queue->execute()
#13 [internal function]: yii\queue\cli\Command->actionExec()
#14 /var/www/html/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array()
#15 /var/www/html/vendor/yiisoft/yii2/base/Controller.php(178): yii\base\InlineAction->runWithParams()
#16 /var/www/html/vendor/yiisoft/yii2/console/Controller.php(180): yii\base\Controller->runAction()
#17 /var/www/html/vendor/craftcms/cms/src/console/ControllerTrait.php(90): yii\console\Controller->runAction()
#18 /var/www/html/vendor/yiisoft/yii2/base/Module.php(552): craft\queue\Command->runAction()
#19 /var/www/html/vendor/yiisoft/yii2/console/Application.php(180): yii\base\Module->runAction()
#20 /var/www/html/vendor/craftcms/cms/src/console/Application.php(91): yii\console\Application->runAction()
#21 /var/www/html/vendor/yiisoft/yii2/console/Application.php(147): craft\console\Application->runAction()
#22 /var/www/html/vendor/craftcms/cms/src/console/Application.php(122): yii\console\Application->handleRequest()
#23 /var/www/html/vendor/yiisoft/yii2/base/Application.php(384): craft\console\Application->handleRequest()
#24 /var/www/html/craft(24): yii\base\Application->run()
#25 {main}
ericdrosas87 commented 6 months ago

To replicate testing @blnkt :

  1. Ensure that the webhook settings in Canto are configured to send requests to the target environment (currently they are pointed at dev)
  2. Add a single image to a single-image DAM field in the Craft CMS and save the entry
  3. In Canto, update metadata - such as adding a tag
  4. Verify the metadata updated in the DB, I used pgAdmin, but you can do the same with the GraphiQL test harness in the Craft dashboard
  5. Delete the image in Canto
  6. Verify the image field is no longer populated in the craft dashboard

Repeat the same steps for a gallery, except you can also test adding/removing an image from the gallery.

khalwat commented 6 months ago

In terms of the Matrix block-embedded fields, I'm not sure whether the migration is faulty, or something went awry in the whole multiple versions / db shuffle we've discussed.

I can take a look at the migration and see if I can replicate it though if you like.

khalwat commented 6 months ago

I just read that the Matrix block type is unused -- that being the case, you could also just delete the Canto DAM Asset field in it, and that'd solve the issue as well.

This is all a result of older versions of the plugin where the columns were text being used, and then the migration issues with new versions of the plugin that use jsonb

Anything here forward will always use jsonb columns. The migration was just there as a convenience to help ease upgrading.

khalwat commented 6 months ago

Doing a quick code review, I don't see anything obviously wrong with the migration in terms of how it handles block-type fields like Matrix:

https://github.com/lsst-epo/canto-dam-assets/blob/develop-v4/src/migrations/m231108_024521_change_to_json_column.php#L65

ericdrosas87 commented 6 months ago

All good @khalwat I appreciate you looking into the migration code, but no need to look into it further. Too many moving parts on our side to identify a single cause for these things, and we have a path forward - but thank you!