search

dotmailer / dotmailer-magento2-extension

The official Dotdigital for Magento2 extension
https://dotdigital.com/integrations/magento
MIT License
48 stars 64 forks source link

[Enterprise] Issue blocking large scale project deployment #628

Closed pavelshulga closed 2 months ago

pavelshulga commented 3 months ago
pavelshulga commented 3 months ago

The issue stems from the way Magento 2 conducts data semanticity updates during the deployments. The core Magento command setup:upgrade is a recommended step during each value-added production release. Due to the way migration business logic is implemented within DD Email module (sits within Setup/Patch and used by both setup:upgrade and dotdigital:migrate CLI commands) in case if receiving Magento installation has large set of historic information, the deployment process extends beyond normal duration and requires special arrangements.

The proposed solution is considered as an initial version and will require improvements in the department of identifying CLI command context.

The context setup:upgrade command takes hours to complete for enterprise level Magento command. Not sustainable for highly automated CI/CD as duration hits all the predefined optimistic max execution time limits (circuit-breaker).

The concept of the solution Prevent full data migration to be triggered during setup:upgrade CLI command, data consistency to be guaranteed using dotdigital:migrate CLI command post-deployment.

sta1r commented 3 months ago

@pavelshulga thank you for this.

I believe the quickest way to achieve what you need here is to insert the data patch into the patch table - which means it will be skipped in the upgrade. 

INSERT INTO `patch_list` (patch_name) VALUES ('Dotdigitalgroup\Email\Setup\Patch\Data\MigrateData');

We are discussing internally whether to remove the MigrateData patch completely, meaning that all merchants will need to run the dotdigital:migrate CLI command as a separate step to set up their integration.

pavelshulga commented 3 months ago

Hi @sta1r

Appreciate the efforts. Understanding the technical prowess of smaller merchants I would assume that in-deployment process data patch is viable option to incorporate. I would suggest exploit the architecture (having DD module *-enterprise) and extending data patch classes specifically in that module to suppress in-deployment data sync process.

Happy to have a call/chat to if need be.

sta1r commented 2 months ago

Closing. Will pick up these ideas on our call but for now I think you have a way forward. MigrateData to be a post-deploy step in the future.