working change listener based cloud migration trigger

[senn]

As I mentioned in #172 and #171 I have created a way of migrating content using a ResourceChangeListener.

I added the code in this PR after changing it a bit to match the AECU way of using a ResourceResolver, etc. It can serve as a guide on how AECU approaches the content migration on cloud.

This service is tested on AEM 6.5, cloud-sdk AEM but also on AEMaaCS. We have verified that the migration is triggered correctly on author when deploying and also on publish after automatic replication in the cloud.

All scripts are executed, taking into account their config, execution history, etc.

How it works:

• service listens for a certain node to be added in path /conf/aecu/trigger. The name of this node is configurable (default=migration-trigger), the type should be nt:unstructured.
• when the resourcechangelistener is triggered scripts are executed using the AecuServiceMBean, so taking into account their history, config, etc.
• the service, however, also triggers a first time when it is activated (on deploy)
• the trigger-node is automatically deleted after migration
• the service is also a factory so multiple trigger-points can be configured (if needed)

To test this service

1. install the bundle
2. provide an OSGi config for the service (the default value will do)
3. create the path /conf/aecu/trigger (sling:Folder) and then create a node migration-trigger (nt:unstructured) below it.
4. This will trigger the migration.

To fully test this on cloud AEM so the migration is triggered on-deploy:

1. create the /conf/aecu/trigger/migration-trigger path (with correct node types) automatically
   • we did this from our ui.scripts or ui.content module but a RepoInit script should work aswell
2. when the AecuMigrationTrigger service activates on author the scripts are executed a first time (so on deploy)
3. this should automatically be replicated to publish
4. the migration can be re-triggered on demand by creating the /conf/aecu/trigger/migration-trigger node with the correct node types

I added my unit tests but since AECU doesn't use io.wcm or junit5-jupiter, I commented them all out. You can still see how it is tested. I just couldn't bring myself to recreating the tests.

[gruberrolandvaltech]

How would this trigger node be created? If it is created as part of the package installation then it might be the case that not all scripts are installed yet.

[senn]

The trigger is indeed part of the package installation. The migration triggers also on activation of the bundle. We haven't had a case where scripts were not installed yet at this point.

I guess you could create a separate package project that installs as last part if you want to make sure all scripts are installed.

[gruberrolandvaltech]

To be on the safe side the scripts would need to be located in /apps so that they are already part of the image build.

One question about the trigger node. Instead of using a node we could also check if AEM runs with a composite node store. This would be the case for for the startup after deployment (image building does not use a composite node store). This way no trigger node would be needed. We simply run the scripts (in /apps) when the instance is up. What do you think?

[senn]

That can work (and I'll try to get it working like this in my project just to see what the effect is).

A possibly problematic scenario that I thought about is:

What if we just want to install a content package locally (like test stuff) without installing the bundles and everything that resides under /apps, /conf etc, and automatically have the migration scripts run on that content?

The normal AECU installation (installhook) allows to do this because the hook can be added to the pom of the content project.

The trigger node approach can also do this if you add the trigger node to your content package so it is present in the content that is deployed. (Maybe the entire trigger node path should be configurable in OSGi and not just the trigger node name.)

How would we do this using the composite node store approach? When deploying content only this is not possible afaik...

[gruberrolandvaltech]

For local testing you could use the GUI tool to execute scripts or JMX (allows script run with history). Not sure how often you would deploy a pure content package that requires AECU scripts. I would say in most cases scripts are related to software changes. But we could also provide both ways to start the execution.

[gruberrolandvaltech]

Btw., AECU tests were migrated to JUnit 5 now.

[senn]

Not sure how often you would deploy a pure content package that requires AECU scripts. I would say in most cases scripts are related to software changes.

I know it's a fringe case, I was just comparing functionality between the different approaches.

[senn]

I changed our code to check for a composite node store and that works fine. But ofcourse now it doesn't work anymore for local cloud environments. We were thinking of a way to still make it work on local cloud envs, and not only on startup but also when installing packages with only content (for example test content). We were thinking of still using the InstallHook here but this is not working because we moved the scripts to /apps/groovyconsole/scripts/aecu. The Installhook only checks /var and /conf.

AecuInstallHook:

/**
     * Checks if the script was not yet executed at all by install hook.
     * 
     * @param path    script path
     * @param history history entry
     * @return not executed yet
     */
    private boolean wasNotExecuted(String path, HookExecutionHistory history) {
        return !history.hasBeenExecutedBefore()
                && (path.startsWith(AecuService.AECU_VAR_PATH_PREFIX) || path.startsWith(AecuService.AECU_CONF_PATH_PREFIX));
    }

I added this in PR https://github.com/valtech/aem-easy-content-upgrade/pull/180

[gruberrolandvaltech]

But if you activate the package hook then the package will no longer work on the cloud. Do you (de)activate the hook via Maven profile?

[gruberrolandvaltech]

Added /apps/aecu-scripts as accepted location for the install hook. See examples-cloud package on branch develop for an example. It allows to activate the install hook based on Maven profile.

[gruberrolandvaltech]

@senn can you update the PR with your composite node store changes? You can use the new AECU_APPS_PATH_PREFIX constant for the path to the scripts.

[senn]

I'll try to update the PR today

[senn]

I added the composite node store code, but as a separate service (it's how we implemented it). I left the resource-change-listener in the code, but if you want I can remove it entirely from the PR.

[gruberrolandvaltech]

Thanks a lot for your update! :) I put some review comments. Once they are done I will do some intensive testing and merge the changes.

[gruberrolandvaltech]

Thanks a lot for the update. Merging now to a feature branch on our side for final polishing.