Change plugin installer to use Eclipse P2 system

[Phillipus]

Summary

• Archi uses Eclipse 4.7
• Archi uses a config.ini file that allows additional plugins to be copied into the "plugins" folder and the Eclipse framework will discover them
• It seems that this no longer works on Eclipse version 4.8 and later

Background

This is the config.ini file that is used now:

eclipse.application=com.archimatetool.editor.app
osgi.bundles.defaultStartLevel=4
eclipse.product=com.archimatetool.editor.product
osgi.splashPath=platform:/base/plugins/com.archimatetool.editor
osgi.bundles=org.eclipse.equinox.common@2:start,org.eclipse.update.configurator@3:start,org.eclipse.equinox.ds@3:start,org.eclipse.core.runtime@start

The interesting line is the osgi.bundles entry and the org.eclipse.update.configurator bundle. This allows additional plugins to be copied into the plugins folder and "discovered".

Using Eclipse 4.8+ this does not work and Archi will not launch. I have experimented with different bundle names but nothing works.

By default, Eclipse will generate a config.ini file that uses the org.eclipse.equinox.simpleconfigurator bundle which relies on a fixed list of bundles:

eclipse.application=com.archimatetool.editor.app
osgi.bundles.defaultStartLevel=4
eclipse.product=com.archimatetool.editor.product
osgi.splashPath=platform:/base/plugins/com.archimatetool.editor
osgi.bundles=org.eclipse.equinox.simpleconfigurator@1:start
org.eclipse.equinox.simpleconfigurator.configUrl=file:org.eclipse.equinox.simpleconfigurator/bundles.info

The dependent bundles (plugins) are listed in the bundles.info file and are fixed. Only these bundles are loaded by Archi at runtime, and any other plugins present in the "plugins" folder are ignored.

This means that if we build Archi using Eclipse 4.8+, we cannot install Archi plugins as we do now. Eclipse docs recommend to use the "p2" update system which is the heavyweight and non-user friendly system that Eclipse uses.

It may be that I am missing something in the config.ini file, but I don't know. I've asked on the forum - https://www.eclipse.org/forums/index.php/t/1097066/

[Phillipus]

See:

https://github.com/diffplug/goomph/issues/69

[Phillipus]

See Eclipse bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=542706

[Phillipus]

See https://www.eclipse.org/forums/index.php/m/1801159/

[Phillipus]

See https://www.eclipse.org/forums/index.php/m/1801199/

[Phillipus]

Note to self - specifying the default config.ini in the Product file that uses the simpleconfigurator in Eclipse 4.7 and manually adding a plugin into the "plugins" folder actually works. But not so in 4.8 and beyond. So we can't even use that.

[Phillipus]

Archi and Eclipse apps don't look so good on MacOs Mojave. Would need to use Eclipse 4.10 to cure this. But in order to use 4.10 we need to figure out how to install plugins using the p2 system and using the config.ini and bundles.info.

[Phillipus]

See Eclipse bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=542706

Sadly, they've marked it as WONTFIX. So we're out of luck using the existing system.

[Phillipus]

At some point we need to move to Eclipse 4.10 (2008-12).

Benefits:

• Latest code fixing some issues and working on latest OS
• Better support for later versions of Java
• Fixes some UI issues on Mac

Cons:

• Existing system of installing Archi plugins will not work, have to use Eclipse's p2 update system
• No 32-bit support - no longer ship 32-bit Linux and Windows versions of Archi
• Possible unknown issues and bugs introduced

[Phillipus]

@jbsarrodie @herve91

I've just spent the last week or so in an Eclipse nightmare of trial and error, no documentation and the usual Eclipse hell and frustration. ;-)

However...

I've developed a new plugin installer and uninstaller based on the Eclipse p2 system. It uses the Eclipse "feature" mechanism to install a given feature into the Archi installation.

We will have to move to this installer in order to use later versions of Eclipse. Older plugin zip files and jars will not work because from Eclipse 4.8 onwards simply dumping a jar file into Archi's "plugins" folder will not work. We're outta luck, guys. Herve's online update will no longer work.

You will have to create a "feature" project for your plugin. We already have these for the some plugins. All you do is export this as "Deployable features" in Eclipse as a zip file.

The great thing is that plugins and features are installed in the user's home directory in ~/Archi4. This means that when there is a new Archi update users don't have to re-install all their plugins.

Also, there is in uninstall plugins wizard.

Any questions?

[Phillipus]

There's a branch "p2" with the code.

[jbsarrodie]

Great...

Any questions?

Yes, does this mean that we could create some kind of web-based plugin distribution system (kind of app store) ? It would be great (for end-user) to simply open a page inside Archi that lists all available plugins, download them and update them when needed. We could have one "official" page for core plugins maintained by Phil and me, and one "community" page (related to archi-contribs) for other plugins (ie: mainly Herve's plugins).

[Phillipus]

Yes. :-)

[Phillipus]

For existing plugin developers it simply means:

• Create a "Feature" project for your plugins
• Export the Feature as "Deployable features" in Eclipse as a zip file

Here's a screeny:

[Phillipus]

Herve's online update will no longer work.

For now. If the local update mechanism is proven to work OK we can then do an online update mechanism.

[Phillipus]

I suggest we move to this system when Archi implements the next version of ArchiMate.

[herve91]

Hi @Phillipus and @jbsarrodie,

I believe migrating my plugins to p2 will be a piece of cake regarding all the work you've done on the core of Archi to adapt to Eclipse 4.8 :-)

Ready when you wish ...

Unfortunately, my Eclipse knowledge does not allow me to help you on a lot this subject :-(

Best regards Hervé

[Phillipus]

Thanks Herve.

Unfortunately, my Eclipse knowledge does not allow me to help you on a lot this subject :-(

My Eclipse knowledge is based on days, weeks, years of frustrating trial and error with no help from Eclipse experts. ;-)

[herve91]

Mine is based on copy and paste from tutos I've found on the net, without always understanding why it was (or wasn't) working :-(

The way I generated JAR files was not even correct ...

A person who is using my plugins hired an Eclipse expert in his team and I expected a lot, may be too much, as nothing has come from this :-(

[Phillipus]

Mine is based on copy and paste from tutos I've found on the net, without always understanding why it was (or wasn't) working :-(

Same here a lot of the time. Eclipse development is woefully under-documented. And don't even bother with the Eclipse forums.

[Phillipus]

I've tested the new plugin install system on Windows, Mac and Linux and several different permutations of installation. It works but in the process of installing, uninstalling and restarting Archi the configuration folder got messed up and Archi would not start. I'm not sure if it was because I was testing, but in case this happens a fix is to delete some files and folders in the config area.

By default this is specified in Archi.ini as:

Windows: @user.home/AppData/Roaming/Archi4

Linux @user.home/.archi4

Mac @user.home/Library/Application Support/Archi4

To clear problems delete the following:

/.config
/features
/p2
/plugins
artifacts.xml

This will remove all installed plugins and their p2 repository information. It's an emergency strategy but worth mentioning until I can see what might cause problems.

[Phillipus]

Install and Uninstall combined into one plugin manager:

[Phillipus]

I've been experimenting with this more and discovered a problem.

The Eclipse P2 system stores plug-ins and information about those plug-ins in the @user.home/Archi4/* folder as specified above, That information is tied to the location of the Archi installation folder.

So, if you move or rename the Archi installation folder (or re-install in a different location) then the P2 plug-ins will not be able to be uninstalled or re-installed properly.

Steps to reproduce:

1. Install Archi with P2 support to installation folder C:\Archi
2. Install some plug-ins using the new system
3. Quit Archi
4. Rename the installation folder to C:\Archi2
5. Re-launch Archi
6. Uninstall the plug-ins - you'll get an exception and the plug-ins cannot be uninstalled properly
7. Try to re-install the plug-ins in Archi using the new system
8. Restart Archi - it now fails to load and Archi cannot start at all

(To fix delete the folders as mentioned above.)

This is a problem for anyone who re-installs Archi in a different folder to the original, or those who use a portable version and move it to a new location.

[Phillipus]

The problem arises because of the separation of the app installation folder and the configuration area.

We define the configuration area in the Archi.ini file. For example, on Windows:

-Dosgi.configuration.area=@user.home/AppData/Roaming/Archi4P2/.config

This is where P2 installs plug-ins and its P2 info.

If, however, this line is removed from the Archi.ini file then plug-ins and P2 info will be written to the app installation folder and there is no problem.

However - Archi would then need write access to the app installation folder and this may not be possible if Archi is installed in Programs folder. Besides, it seems silly to have this information tied to the app installation folder. Also doesn't work if the app is shared among users on a network install.

[Phillipus]

I asked about this in the Eclipse forums:

https://www.eclipse.org/forums/index.php/t/1097883/

I've done what I can given my limited knowledge of P2, but we need some advice on this issue from an Eclipse P2 expert, otherwise we won't be able to use the next version of Eclipse and support Archi plug-ins.

[Phillipus]

Re-wrote the P2 code to support the "dropins" folder installation of plug-ins. Got it working but came across the same problem if the Archi installation folder is moved/renamed and then a plug-in is uninstalled.

Then I found Eclipse itself has the same problem when using the "dropins" folder. Steps to reproduce are at https://www.eclipse.org/forums/index.php?t=msg&th=1097883&goto=1803694&#msg_1803694

Then I remembered that the Modelio tool uses the "dropins" folder approach and it too has the same problem:

1. Install Modelio into folder "Folder1" with their ArchiMate plug-in copied to the installation's "dropins" folder.
2. Run Modelio to check the plug-in is installed and then quit it.
3. Rename the folder to "Folder2".
4. Run Modelio to check the plug-in is still nstalled and then quit it.
5. Delete the ArchiMate plug-in from the "dropins" folder.
6. Run Modelio...Boom! Won't launch.

[Phillipus]

After another couple of 12-hour days doing the usual Eclipse trial and error / no help routine I may have come up with a workaround to the moved/renamed installation folder issue.

Cause

If plug-ins are installed/uninstalled using the Eclipse p2 "dropins" folder method and the Archi installation location is then moved/renamed, various files in the configuration area reference the previous installation location and Archi will not load. Same for other Eclipse based apps that use P2 (Modelio)

Findings

If certain files and folders in the configuration area are deleted prior to launching Archi then it will load OK. These files/folders are:

@user.home/AppData/Roaming/Archi4/p2
@user.home/AppData/Roaming/Archi4/artifacts.xml

And all files and folders in

@user.home/AppData/Roaming/Archi4/.config/

Except for this folder which is already cleaned with the -clean option:

@user.home/AppData/Roaming/Archi4/.config/org.eclipse.osgi

Workaround

I have written a class called WorkbenchCleaner that is called when Archi first starts. It simple deletes the above files and folders.

Seems a bit crazy to have to do this but I can't find another way to do it.

[Phillipus]

However, this may not work forever:

"Do realize though that officially the dropins is deprecated and not recommended."

https://www.eclipse.org/forums/index.php/m/1800936/#msg_1801166

[jbsarrodie]

Does your workaround also solves the same issue for plugins installed through p2 (not dropin) ? My assumption would be that they both share the same root cause, so solving one might solve the other...

[Phillipus]

Does your workaround also solves the same issue for plugins installed through p2 (not dropin)

No.

The P2 method actually installs an Eclipse "Feature" (Feature + Plugins Jars) and various entries in the p2 repository files (a.k.a "nightmare"). If you delete any of the files or folders mentioned above then you will mess up the plug-in installation.

My conclusions are:

1. Unless an Eclipse expert can tell us otherwise, I think the move/rename installation issue is a fault with the Eclipse p2 system.
2. Using "dropins" works for now and is less deprecated than the system we use right now (Modelio uses it). If my kludge proves to be OK, we can use that for a couple more years.
3. I have a beer and I'm going to drink it...

[jbsarrodie]

Ok. I guess point 3 is the most important ;-)

[jbsarrodie]

[Phillipus]

And the latest issue when using the latest version of Eclipse and P2 is this one:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=546384

It takes 90 seconds to launch Archi. :-(

Edit - this only happens sometimes in development mode.

[Phillipus]

Just a reminder why we delete the p2 files at startup:

• The Archi installation directory in some cases will be read-only (Windows installer)
• Therefore we need to set the config and p2 data locations to a writeable location (user.home sub-directory)
• The user might move the Archi installation directory, or might install it to a different location
• The config and p2 files will now reference the absolute location of the moved Archi installation directory
• If the user moves the Archi installation directory one more time then the config and p2 files will reference the previous Archi installation directory
• Archi will not launch

[Phillipus]

There is one more factor that contributes to this problem:

• If the "dropins" folder is in the Archi directory and the Archi directory is renamed or moved

[Phillipus]

More experiments show that the main problem is caused by this:

• If the "dropins" folder is in the Archi directory and the Archi directory is renamed or moved

Analysis

The config.ini file that Eclipse creates in the configuration area in user.home (not the one in the "configuration" folder) should look like this:

osgi.sharedConfiguration.area=file\:configuration/
eclipse.p2.data.area=@config.dir/../p2/

But after an Archi installation folder rename or move (for example to "D:/Archi") it adds a line like this:

osgi.bundles=reference\:file\:/D\:/Archi/dropins/com.archimatetool.script_0.5.0.201907121721.jar@4,reference\:file\:/D\:/Archi/dropins/com.archimatetool.script.commandline_0.5.0.201907121721.jar@4,reference\:file\:/D\:/Archi/dropins/com.archimatetool.script.premium_0.5.0.201907121721.jar@4,reference\:file\:/D\:/Archi/dropins/org.archicontribs.modelrepository_0.5.2.201907081356.jar@4,reference\:file\:/D\:/Archi/dropins/org.archicontribs.modelrepository.commandline_0.5.2.201907081356.jar@4

These are absolute paths to plugins in the Archi installation "dropins folder.

That's OK as long as you don't move or rename the Archi installation folder again. But if you do, those absolute file paths are not updated and that causes Archi not to start (same with Modelio).

That's why Archi deletes files in the config area (and p2 folder) to clean up this mess.

[Phillipus]

So, if the org.eclipse.equinox.p2.reconciler.dropins.directory property is set and that "dropins" folder is used this should not be a problem, and the config files don't need to be deleted at startup.

But if a local "dropins" folder is used in the Archi installation folder and that is moved or renamed then there will be a problem and the config files need to be cleaned at startup.

For a portable installation where the "dropins" and config files are local to the Archi installation folder this is the recommended Archi.ini for Windows and Linux (Mac Archi.ini is similar but has more entries):

-clean
--launcher.defaultAction
openFile
-vmargs
-Dosgi.requiredJavaVersion=1.8
-Dosgi.instance.area=/pathToInstanceArea
--add-modules=ALL-SYSTEM

Edit 13 Jan 2021 -

On Mac the path to a plug-in in the dropins folder is written into bundles.info as a relative path like ..\..\..\..\pathto_plugin so if the App is moved then that relative path is no longer valid.

Added a -cleanConfig option to Archi.ini to delete all these files after the app is closed.