v.1.8.4 - Sync between rep:permissionStore and repo doesn't work for large session.save() on the AEM 6.1 SP1 with MongoDB

[mtstv]

AC Tool rel. v.1.8.4. Sync for permissions between rep:permissionStore (/jcr:system/rep:permissionStore/crx.default) and repository doesn't work correctly for large number of nodes during session.save() on the AEM 6.1 SP1 with MongoDB. There is a problem for example with 7400 jcr paths where permissions must be created.

During the AC Tool execution the part of permissions will be saved for some groups under /jcr:system/rep:permissionStore/crx.default/<group-name>/<generic-node-name>, but some part of permissions will be not saved under rep:permissionStore, even though all permissions are correctly set in the repository itself. I can confirm that all ACEs are in repository as they were configurated in yaml files, but not every permission has been synchronized with /jcr:system/rep:permissionStore.

What I did 1) I added session.save() for every 100 nodes and the SYNC works now for new nodes, but not for already UNSYNC permission nodes. 2) To repair the UNSYNC state I added also session.save() for every 100 nodes during the deletion of principal ACEs which started before the installation of permission nodes.

After this 2 changes everything works now.

I created my own branch just to test and validate the solution https://github.com/mtstv/accesscontroltool/tree/develop-nohistory-100save-unsyncrepair

What do you think about it and do you have somthing against if I add it for the next release v.1.9.2?

[kwin]

Just adding intermediate save()s makes rollback a lot harder. In the past it was supposed to work in an "all or nothing" manner. That will change with intermediate save()s as we cannot easily rollback.

[mtstv]

But there is already intermediate persisting for principal removing, principal installation and ACE creation.

• removeAcesForAuthorizables(history, session, authorizablesToRemoveAcesFor, repositoryDumpAceMap);
• installAuthorizables(history, authorizableHistorySet, authorizablesMapfromConfig);
• installAces(history, session, aceMapFromConfig);

How I understand in case of exception during the ACEs installation with intermediate persisting the all earlier existing ACEs of principals must be restored again, means must be collected into HashSet before they deleted. It must be added to code.

But in case if history and rollback are not needed at all, because we can simply install the early configuration, we can actually do the intermediate persisting for ACEs. What do you think if we can allow to disable the rollback and intermediate persisting and make them configurable over osgi?

[ghenzler]

@kwin I was involved a bit more in the discussion between the client and Adobe and I know where the pain comes from. The root cause is in oak (most likely https://issues.apache.org/jira/browse/OAK-5557), but @mtstv's fix was good to at least get everything up and running again.

I'm doing quite a bit of refactoring for #146 and #127, let me check if a configuration option for intermediate save would make sense ( @kwin all large installations with MongoDB and AEM 6.1/6.2 can run into this problem, it would be good to have an easy workaround at least... rollout would be just disabled then if something goes wrong, but as it stands today the rollback is only used for JMX based installation anyway and not for install hooks)

[ghenzler]

This will be fixed by #149

[ghenzler]

@mtstv please note that the way how acls are deleted has changed completely - up front only the ones that are outside the "configuration path space" are deleted, the ones for configured paths are deleted for each path individually and hence the intermediate saves take effect here as well

[mtstv]

Hi Georg, I just worried about the case which repairs the UNSYNC state between repo and permissionStore. Do you mean that the setPolicy(aceBean.getJcrPath(), acl); at position 302 in the following java class will touch/overwrite the ACEs in any case, also if new ACEs completly equal to the already existing ACEs in the repo?

/accesscontroltool-bundle/src/main/java/biz/netcentric/cq/tools/actool/acls/AceBeanInstallerImpl.java

    private void install(AceBean aceBean, final Session session, Principal principal,
            AcInstallationHistoryPojo history) throws RepositoryException, SecurityException {
...
        acMgr.setPolicy(aceBean.getJcrPath(), acl);
    }