Potential 2.0 version discussion

[konadave]

The main weakness of the current overrides extension is the fact that none of it is automatic. The person upgrading Civi needs to remember to take a snapshot before the upgrade so that it can detect changes properly. And then after the upgrade they have to go manually look to see if there were changes. Then deal with it. It should be easier. Here's my thinking on a 2.0 version of this extension.

On this extension install:

• compile/save list of overrides for all extensions found on the system, enabled or not
• display warnings for any conflicting overrides found

On other extension install/enable/upgrade:

• compile list of overrides for extension, list them in the table with the info.xml details, include a help link explaining what that means
• if any override is an override in an aleady enabled extension, disallow, provide disable link for conflicting extension
• using a yet unresearched/unidentified method (maybe a github/lab api or exec-ing git), determine date of last modification in core, relative to installed version of CiviCRM, for each override. display warning/potentially disallow if any modified more recently than the release date of extension
• generate and save diffs for each override post install

On other extension uninstall/disable:

• no special action required

On Civi upgrade:

• check all overridden files to see if they have changed, display warnings if so

Extension UI:

• issues that need addressed will show up in system status
• overrides page similar to current, listing which extensions override and what they override, plus...
• unified difference view showing changes in override
• when core file changes, integrate those changes into the diff view, and...
• provide some sort of means to work those changes back into the override
• you can check out compare default in better message templates to get an idea of what this might look like/how it might work

Assume overrides are based on compatibility version in info.xml:

• provide the means for the extension to define the version per override, maybe via info.xml or a hook

Notify maintainer when an override is changed in core:

• send email to maintainer in info.xml, but then maintainer gets an email from every site that upgrades?
• communicate with a companion extension installed on civicrm.org that would filter these notifcations?
• post an issue to their repo?
• should it be an opt-in in the info.xml?

Hooks and stuff:

• hook_civicrm_pageRun/buildForm to handle pre action checks
• would be nice to take other steps at completion of action...
  • hook_civicrm_postProcess is not called, so need another way to detect extension changes
  • hook_civicrm_post called for entity == extension and action == delete during uninstall, but that's it
  • hook_civicrm_postSave_civicrm_extension is not called
  • was wishful thinking api.System.check would be called after changes, triggering hook_civicrm_check, but no
  • extension lifecycle hooks are only called on the extension in question, not others
  • listening to civi.api.respond can see 3.extension.install and 3.extension.enable, not sure yet about upgrade, but it does not see disable or uninstall. if upgrade is seen, this may be enough, however...
  • enabling via cv did not use the api so was not seen
  • hook_civicrm_navigationMenu in theory (i.e. I haven't tested yet) should be called after the status of an extension change, so it may be suitable as a trigger to identify overrides and their diffs
  • but I think hook_civicrm_somethingUpgraded that gets passed the key and current state would be nicer, assuming core team is open to minor changes in order to support this
• hook_civicrm_check will provide the "conflicting overrides" and "core was updated" warnings for system status

[konadave]

Assume overrides are based on compatibility version in info.xml

Actually, if an extension has an override it should be making sure it's up to date with latest when it's released, so assume based on release date instead.

using a yet unresearched/unidentified method (maybe a github/lab api or exec-ing git), determine date of last modification in core, relative to installed version of CiviCRM, for each override...

function overrides_getAllReleases() {
  $notes = file_get_contents('https://raw.githubusercontent.com/civicrm/civicrm-core/master/release-notes.md');

  $versions = [];
  preg_match_all('/^## CiviCRM (.+?)$/m', $notes, $versions);

  $dates = [];
  preg_match_all('/^Released (.+?)$/m', $notes, $dates);

  return array_combine($versions[1], $dates[1]);
}

...will produce an array where the keys contain the version number and the values contain the release date back to 4.7.14. We can use the same base URL to download the correct files to compare with.

For example, let's say an extension released Jan 1st 2023 contains an override of CRM_Utils_System. https://raw.githubusercontent.com/civicrm/civicrm-core/5.56.1/CRM/Utils/System.php to determine what's been overridden by the extension, and https://raw.githubusercontent.com/civicrm/civicrm-core/master/CRM/Utils/System.php to compare against latest to see if changes have since been made.

Downloading via raw avoids the need to create access tokens to use an API. This might be good enough, we'll see.

[konadave]

Just an FYI for the zero followers...

All the hope of preventing conflicting extensions from getting installed probably isn't worth the effort, it's too easy to bypass. One could install via cv, or uninstall the overrides extension first, or remove the override initially to get it to install then add it back in or otherwise add an override after the fact.

The idea of reacting to install/enable/disable/uninstall and taking/saving diffs and whatnot is also unnecessary. It's going to do it's thing on demand; once a day as part of api.System.check (i.e. system status) and when directly accessing the overrides UI.

Automated communication with maintainer is also probably over the top. Will probably just provide a mailto: link to the maintainer somewhere.

A new 2.0 beta is about ready for testing. Much smarter and no direct initiation from user required. Will likely drop in a few days.