Upgrade Papyrus version

[J17359]

Issue and tracking information

Developer's time Estimated effort to fix (hours):

Developer's Actual time spent on fix (hours)

Issue reporter to provide a detailed description of the issue in the space below

Ideally, upgrade to the latest version of Papyrus, but I'll leave that up to you to evaluate. I'd say at least 6.x. The version CX currently delivers is 4.8.0 (2020-06). There are two newer major releases 5.x and 6.x available on their home-page.

[eposse]

A quick status report.

I have been finally able to make a target platform based on the Papyrus 6.5 (2023-06) SDK, and after updating a lot of dependencies on manifest files, I am able to compile all (except for Papyrus Compare) within Eclipse. I have not yet tested it.

Next steps (not necessarily in order, as dependency management may require updates):

1) Try to build it with Maven. 2) Adjust the RCP product as needed by the new dependencies. 3) Test. 4) Include and adjust Papyrus Compare.

[eposse]

An update.

I got it building with Maven, including the Papyrus patch, but without Papyrus Compare, and updated the RCP product, although I might need to streamline the list of dependencies in the RCP product, as it seems to be getting too large, but that's a secondary concern at the moment.

I'm going to be testing now.

[eposse]

Some important info:

After building, I discovered that the RCP still contains two versions of log4j 1.2, in addition to log4j 2:

org.apache.log4j_1.2.15.v201012070815.jar
org.apache.log4j_1.2.24.v20221221-2012.jar
org.apache.logging.log4j_2.17.1.v20220106-2156.jar

Version 1.2.15 could possibly be explained by one of our existing plugins still having that dependency, but I didn't understand the presence of the 1.2.24 version, until I looked at a plain Papyrus install, and it turns out that plain Papyrus is the one that includes it, in addition to 2.17.1.

So there doesn't seem to be a clean way to simply get rid of log4j 1.x by upgrading (or otherwise)

I will investigate to see if perhaps I can find which plugins are dependent on 1.2.x, and hopefully I can patch them. The problem is that it may be a third party dependency, like EMF Compare, or worse, Xtext. This is very, very difficult because these, as well as Papyrus itself, have a very large code base.

[eposse]

And more info about this.

I have tried to install several variants of Eclipse to see which ones come with versions of log4j, and in particular I tried the following ones which are all relevant as they all include features that are included in Papyrus CX. They are all based on the latest release, which is 2023-06:

Eclipse RCP................................... Includes log4j ........ log4j versions
-----------------------------------------------------------------------------------
Eclipse Platform (minimal Eclipse install).... No
Eclipse for C/C++ Developers.................. No
Eclipse for Java Developers................... No
Eclipse for Java and DSL Developers........... Yes .................... 1.2.24
Eclipse Modeling Tools........................ Yes .................... 1.2.24
Papyrus....................................... Yes .................... 1.2.24, 2.17.1

So clearly version 1.2.24 comes from components in both the DSL package and/or the modelling package and Papyrus includes a great deal of both of these. These share a lot in common. My guess is that Xtext is the main culprit. I have been following details of Xtext development and they have been in trouble for at least two years with a lack of developers, so upgrades there are very unlikely. This is a big problem, because Xtext is huge, with an extremely complex code base.

I'll try to confirm which features and plugins exactly require 1.2.24.

EDIT: another relevant link, Message #1851012 from the Eclipse forums. Further information supporting the claim that the org.apache.log4j_1.2.x bundles are not the ones with the vulnerability, in spite of the version number.

[eposse]

My fears proved to be correct. Xtext (the latest version) depends on org.apache.log4j 1.2.24. And it's not just a few plugins that could be easily patched. It is all over the place: all 4 features including the "top-level" features and 71 plugins!!!

Removing this dependency will be very, very, very difficult, and you should take into account that we are not familiar with its code-base.

I will try to see how many of those plugins are actually included in Papyrus, in case its a smaller number that I can patch.

But either way, this is going to take a lot longer that we may have thought.

Xtext has a closed issue on migrating to log4j2 but the discussion there is that they do not have the capacity to do so. There are some suggestions on what to do in that discussion, but I don't know if they would completely remove the 1.2.x versions. I'm reading on them. The discussion is found in this link. EDIT: Note that the issue was closed, so basically, there is no movement from Xtext on this.

[eposse]

An update:

I am working on trying to patch Xtext to use the latest version of log4j 2.

Plain Papyrus has 18 Xtext plugins. Unfortunately, there is no feature in plain Papyrus that includes these plugins explicitly. They are brought in during the build because they are listed as dependencies in some of Papyrus plugins MANIFEST.MF files. Now, this is a problem (that there is no explicit Papyrus feature with them) because it means there is no feature to patch. In an Eclipse product, if you want to patch a plugin, you can only do it by patching a feature that contains the plugin. Now, in Papyrus CX, we actually do include four Xtext features, so we can patch those. The problem is that it requires patching 36 plugins rather than 18. So it's double the work.

The good news is that I have been able to patch one Xtext plugin. 35 to go. Having done one, the others should, in theory, be easier, and a bit faster, but it's still quite a lot. And then after that, I will likely need to patch several other features, likely OCL, Xtend, Xpand, EMF Compare and maybe others. So it will take time.

[emammoser]

Thank you for your persistence, we appreciate it. We are still trying to get direct answers from our security team on what is/isn't allowed. We realize that Log4j 1 has had numerous updates and CVE's discovered and fixed, however, it is my current understanding that anything log4j 1 is not going to be allowed. As soon as I get an answer different from that I will let you know, but I do think we should continue working on replacing log4j1 entirely, as you have been.

Is there any chance that once you do manage to patch these plugins, you could push these patches to the official branches for Xtext? If we can help the open source community version patching them locally before building and packaging, that would be our preference.

[eposse]

I was thinking exactly that, to push my updates to the official Xtext repo. However, the full Xtext includes something like ~70 plugins, so patching all would still be a bit of a tall order. Furthermore, I'm doing the updates based on the latest release, (used by Papyrus) which is 2.31.0, but they are working on 2.32, so it might be a bit of a catch up game. Of course, I could try to update all their plugins if you wish me to do so, but that would take time away from dealing with CX issues directly, and I will still need to patch other things like OCL, Xtend, etc. It's up to you to set the priorities. But I might still do a pull request in their GitHub repo, so that at least others can continue.

[emammoser]

Yes, I agree with you completely. We can push up the changes in an attempt to contribute but let's not put any extraneous effort into getting our changes accepted by their product.

[eposse]

An update:

I have patched almost the entire Xtext Runtime feature, 9 out of 10 plugins. I'm working on the 10th, which is the trickiest, as it deals directly with Log4j' extension mechanism, which changed a lot from Log4j 1 to Log4j 2. After that one, I think the remaining 26 Xtext plugins (in 3 features) will be a lot easier.

[emammoser]

Glad to hear you are making good progress. Thank you for the update.

[eposse]

A status update:

• I've migrated a total of 21 Xtext plugins from Log4j 1 to Log4j 2; this includes all the plugins in the main Xtext features included, but not the Xtext features included in the Xtend features
• I've migrated 3 Xtext plugins from Apache Commons Logging (JCL) to SLF4J
• I've migrated 5 CX plugins from Apache Commons Logging (JCL) to SLF4J

The last two need a bit of an explanation. Many plugins, both ours and our dependencies use "Apache Commons Logging" (previously known as Jakarta Commons Logging or JCL), which is provided by the org.apache.commons.logging plugin, provided by the Orbit repository, and unfortunately, this plugin depends on Log4j 1.2.x. SLF4J is a Java facade for several logging services, and it is easy to migrate from JCL to it, and it doesn't depend on Log4j 1.2.x. In fact, we could also migrate all Log4j to SLF4J instead, but at least for now, I'm trying to do whatever causes the least possible disruption.

[eposse]

Another update.

I've run into a big hurdle that has complicated the process of patching. Normally the dependency on log4j is declared in plugins, and those are the once I'm patching. But features can also declare hard requirements, and there are several features, including the core Xtext feature that declares log4j 1.2 as a hard requirement. These hard requirements are installed when the feature is installed. This is a problem, because the typical way to patch Eclipse products is to define a "Feature Patch" that specifies which feature is being patched and the plugins with the updates. But they way this works is that when Eclipse installs the feature patch, it first checks that the original feature is installed, and if not, it installs it, and then installs the patch, replacing the old versions of the patched plugins with the new ones included in the feature patch. But since it installs the original feature first, it also installs its required features and plugins, which in our case is the old Log4j 1. So this means that we cannot use the usual feature patching mechanism. Instead what I am trying is to copy the original feature entirely, removing the hard requirement to Log4j 1, and include this updated feature instead. It has been very difficult to build the product this way, but it looks like I have been able to build one with the new versions of the features.

Nevertheless, there are still many other features and plugins that have the log4j 1 dependency, including, critically, Xtend. This is critical because Xtend is the language in which the code generators are written. So as a next step, I'm going to try to remove the dependency in Xtend. After that, other dependencies such as MWE and MWE2 (the "Modeling Workflow Engine") also depend on it, and both Xtend and Xtext depend on MWE(2).

[emammoser]

Thank you for this update. It looks like you have a path forward, even if it is difficult. I wish we could say that we didn't need this improvement, but we recently received security scans that flags all instances of log4j1 as critical/severe.

Now that you have brought up Xtend, I remembered you stating in a different post that the version of Xtend that we use is partially the reason for the generator slowness and that there is a newer version that is "faster". Does this newer version have log4j2 or log4j1 dependencies? I do not want to increase the scope of this ticket to also upgrade Xtend unless it completely removed the log4j1 dependency

[eposse]

Well, it is not as simple as upgrading the Xtend version. The language (Xtend2) is actually different (from old Xtend). It has a different syntax, so it would require rewriting the generators. While doable, it is not trivial and would take time (weeks). Furthermore, Xtend2 still depends on log4j 1, so just upgrading wouldn't get us closer. So I would defer upgrading the code generators and deal with that in a separate issue later.

In any case, I need to patch both Xtend and Xtend2, because it turns out that we do use Xtend2 as well, mainly in some utilities.

So for now, let's stick with just patching all, both Xtend and Xtend2.

[emammoser]

Sounds good to me, thank you for the information

[eposse]

A status report:

I have now patched almost all of the Xtext plugins and features (I think I'm missing one), and all of the Xtend2 feature and plugins.

Next, finish any pending Xtext plugins, and patch the old Xtend 1 features and plugins.

[eposse]

Status report:

I've finished patching all Xtext, Xtend 1 and Xtend 2 features and plugins that used Log4j1 or JCL (which uses Log4j1).

Next in the list: Xpand.

[eposse]

An update.

Xpand was very small and simple and I've patched it.

I encountered a difficulty with part of "ECF" (the Eclipse Communications Framework). One of the core Eclipse features "org.eclipse.equinox.p2.core" requires a features called "org.eclipse.ecf.filetransfer.httpclient5", which itself requires JCL (org.apache.commons.logging), which requires org.apache.log4j, i.e. Log4j 1.

This means that the requirement is at the very heart of Eclipse itself.

It looks like I was able to patch org.eclipse.ecf.filetransfer.httpclient5 as I have with other features. But being required by a core feature makes me a bit nervous. Fingers crossed, it will work.

Next, I've found that "Ant" (a Java Building tool) is required by many plugins, and it itself depends on Log4j1. I'm trying to patch that now, although that is a standalone plugin, without a feature to contain it, so I'm not sure how that will work. Hopefully it will.

[eposse]

I ran into difficulties building Ant, because itself is built in a very different way and has dependencies which I haven't been able to find.

I'm going to leave that on hold for a while and move onto a different component and circle back to it later.

[emammoser]

Is Ant itself installed with papyrus? Or just used to build other tools?

Either way, it is depressing to see that Eclipse and its foundational software isn't being maintained as much as we would have hoped. Log4j1 has been EOL since 2015.

[eposse]

Unfortunately Ant is installed with Papyrus. I'm not really sure why, but it is required by a lot of plugins, including some required directly by CX, like org.eclipse.emf.mwe.core.

I agree about the state of Eclipse and Papyrus updates. The only thing I can say about this is that it looks like the many projects that comprise Eclipse, such as Xtext and many, many more, do not have the capacity to make the updates, and, as the previous links I've provided suggest, the actual Log4shell vulnerability, which is the most famous, affected a version of Log4j 2 rather than Log4j 1, although Log4j 1 has its own vulnerabilities. So perhaps because of this, it has not been seen as a bigger priority by the different teams working on Eclipse projects. Also, some of these projects are not even Eclipse projects, like Ant.

I also think that when projects get too big, like many Eclipse projects, it becomes harder to upgrade dependencies as time passes. A component depends on another component, and then someone builds another to depend on it, and so on, so old dependencies linger for a long, long time. And since in this case the problem dependency was a logging library, it was considered low priority. But I think this sort of problem of legacy components lingering for a very long time, is not exclusive to Eclipse or Papyrus, but to software in general. As software becomes larger and more complex, it becomes more difficult to upgrade the older parts.

Anyway, today I'm working, not on OCL as I said, but on "Nebula", another set of plugins required by Papyrus. I decided to work on this one first, because it is smaller, just four plugins.

[eposse]

I've pushed my current progress to a PR #480 so that we have the code in the repo. Obviously it is far from ready, but it's better to have it in GitHub than just in my development machine.

You may even give it a try to see if you find unexpected behaviours. I imagine there will be some issues.

As for progress, I've finished with the Nebula/Nattable component. I'm looking to decide which one to update next.

[eposse]

A status update:

I decided to look into the dependencies in Papyrus itself before other components like OCL or MWE.

There are 29 Papyrus plugins with a dependency on log4j1, and one feature that lists it as a hard requirement.

I decided to start replacing the feature (org.papyrus.infra.gmfdiag.feature) by removing its hard dependency. But as I've mentioned before, this cannot be done by a "Feature Patch", but instead providing a copy of the original feature with the hard requirement removed. This then creates a problem, because during the build, maven tries to resolve feature's requirements, but if you have modified a feature, you get it with a new version number so when the build is trying to establish the requirements of other features that include the modified one, it fails to find it. More concretely, we modified the org.eclipse.papyrus.infra.gmfdiag feature (version 3.0.100.qualifier), so we get it with a new version (3.0.101.qualifier), and then, during the build, maven finds feature org.eclipse.papyrus.infra (version 3.2.100.qualifier) which requires org.eclipse.papyrus.infra.gmfdiag 3.0.100, so it doesn't find it. This means we have to create a new version of org.eclipse.papyrus.infra (3.2.101.qualifier) that contains the new one. But then this fails because the top feature org.eclipse.papyrus.sdk (6.5.0) expects org.eclipse.papyrus.infra (3.2.100.qualifier), so we have to modify that one again. But then this results in a new problem, with the build complaining that it cannot resolve a plugin from one of the feature patches.

So this suggests that the only way forward is to ditch entirely "feature patches" and only provide explicit new versions of features that we need to modify. I think this will be possible, and it may actually simplify the build since we now have to build feature patches separately because of Tycho limitations.

Hopefully this approach will work.

[emammoser]

Thank you for the update. The issue with the feature patches makes sense. A change this large was bound to cause more and more issues the more we unraveled it, so the fact that we can't hack this altogether with patches isn't surprising.

[eposse]

An update:

I've upgraded the Papyrus feature that required log4j 1 as well as the other 4 features that depended on it.

I've also upgraded 6 of the 29 Papyrus plugins that depended on log4j1.

[eposse]

I've upgraded today 19 more Papyrus plugins, for a total of 25 out of 29.

[eposse]

All Papyrus features and plugins directly using log4j have been patched. Next on the list are MWE2 and OCL. MWE2 is smaller, so I'll start with that one.

[eposse]

I'm done with MWE2. I'll start with OCL. That's another big one. Looks like it's 26 plugins and at least 2 features.

[eposse]

A quick update.

As I mentioned in my previous message, I started patching OCL. Unfortunately I am currently stuck unable to build it.

There seems to be a major issue with building OCL related to its widespread use of some Eclipse Java annotations, specifically @NotNull and @Nullable. The compiler throws a lot of errors. I am trying to figure out how OCL is built in isolation. But the problem seems big and long-standing. There are some Eclipse Bugs about it such as Bug 526011 and Bug 564014. I am going through these bugs and trying to see if there are any workarounds.

[eposse]

I've made some progress. I was able to build (most of) OCL in isolation, in particular the version that we need. Now I have to figure out how to integrate that build into ours.

[eposse]

Some good news. I was stuck unable to build a single OCL plugin within our build, but I was finally able to do it. There are about 25 more OCL plugins, but having done the first one means that the others will proceed more quickly. I'm guessing it will take me two more days.

[eposse]

And to add to the previous comment, I've finished 6 out of the 26 OCL plugins today.

[emammoser]

Thank you for the updates. In a rough estimate, how many more plugins do you think you have left to go? Just curious.

[eposse]

For OCL, I have finished 10 plugins and 16 more to go. After OCL there are not too many, but at least one (Ant) is tricky.

The remaining ones are:

org.apache.ant org.apache.httpcomponents.httpclient org.apache.xmlgraphics org.eclipse.mwe.core org.eclipse.mwe.utils org.freemarker

So 6 more, but as I said, Ant is tricky. I tried it earlier but it has a dependency on what appears to be a proprietary plugin from IBM which I can't find anywhere. Unfortunately, a ton of plugins require Ant, including plugins from Papyrus, EMF, UML, Xtend, MWE, and even Eclipse itself.

I'm guessing the other Apache plugins and Freemaker are going to be tricky too, as they are not provided by Eclipse projects, where the build is generally easier to reproduce and the dependencies are generally easier to track.

[eposse]

A status update:

I finished MWE without problems. Now I moved to org.apache.xmlgraphics, but ran into an unexpected issue. Patching the plugin itself was easy, but it turns our that a core feature called "GMF runtime third party dependencies" has a hard dependency on the version of org.apache.xmlgraphics, which meant I had to patch that feature. But patching that one then causes a build problem because another feature "GMF core" depends on that one, so that has to be patched too, but the GMF repository doesn't seem to have that feature. Nevertheless, I got that repository and tried to build GMF Runtime in isolation, but apparently that requires Java 17, however that is incompatible with the version of Maven that we are using (3.6.3). I'm experimenting to see if we can move to a more recent Maven version, but at least OCL declared Maven 3.6.3 as a requirement in its build. I'll try to see if I can make it work with more recent Java and Maven versions.

[eposse]

Another update:

It seems like I can patch the GMF features without having to wrestle with the Java and Maven versions. However I encountered another problem. It turns out that one of the very core features of Eclipse itself (e4.platform.ui) has a hard version dependency on apache.xml graphics. Trying to patch that one, revealed that other core Eclipse features depend on it, namely e4.rcp and rcp. I am now trying to figure out if I can patch them. But the conclusion so far, is that the dependency on log4j 1 goes very, very deep into the core of Eclipse itself. And this has a big consequence: it may be very difficult, in the future, to upgrade to newer versions of all these dependencies, from Papyrus to Xtext to Xtend to OCL to Eclipse itself.

[jwillemsen]

Maybe contribute all patches upstream so that they are hopefully integrated upstream so that a future upgrade can be done more easily?

[emammoser]

@eposse We agree, this effort has certainly exposed just how deep log4j1 runs in eclipse as well as how ill maintained eclipses dependencies run. We are stuck between a rock and a hard place at the moment. I will reach out to my management to see if they have any thought for how we should proceed.

@jwillemsen We discussed this earlier in this Github issue, although that was before the scope grew to this level. If we continue to move in this direction, we will definitely attempt to have Ernesto contribute these to the open-source community.

[jwillemsen]

@j26151 Sorry, missed the earlier discussion

[eposse]

Indeed, it's a difficult situation, but I just want to clarify that it still looks doable, patching the Eclipse core features that is.

There is something to be said about maintenance of dependencies in Eclipse projects though. Eclipse itself consists of many, many independent projects (see https://projects.eclipse.org/) with different teams, and most don't have a lot of developers and have limited funding, if any. They are open source and most developers are volunteers. They don't all have the same priorities either. How dependency management is done, is up to each project, so there is bound to be all levels of quality and maintenance there. In fact, that's what I have seen in all the projects that I've been patching. And with the log4j issue in particular, I think that these aspects, combined with the fact that the main vulnerabilities affected earlier versions of log4j 2 rather than log4j 1 and the ones that affected log4j 1 affected web applications rather than desktop applications and were considered much lower risk, meant that making such upgrade was never a high priority. So I think it is understandable that dependency maintenance in Eclipse projects is not the same as it would be in a closed product developed by a single organization.

[eposse]

Today's update:

While patching Eclipse features I got contradictory build requirements with OCL, but I managed to get it working again by upgrading to the latest version of ECJ (Eclipse JDT's own Java compiler) which is required by OCL. This is required because OCL uses some ECJ-specific annotations and won't compile without them.

After that, I'm still not able to completely build because org.eclipse.e4.rcp depends on a specific version of org.apache.xmlgraphics, so I needed to patch org.eclipse.e4.rcp, but in turn, org.eclipse.rcp depends on org.eclipse.e4.rcp, so I needed to patch that. The problem is that I did not know where is the source code for org.eclipse.rcp, as the structure of the nine core Eclipse Platform git repositories is rather strange, but I finally found it. Now I'm in the process of patching that one. So hopefully that will be the end of this one.

Update: I jumped to conclusions. I was able to get the latest version of the rcp feature, but not the one that matches the version we need, and switching to the relevant branch, that feature is gone from the source, so I'll have to keep looking to find the correct version.

[eposse]

Finally I found the org.eclipse.rcp 4.28 source. It turns out that Eclipse went through a reorganization of their git repositories and this feature and version were on a now deprecated git repo (https://github.com/eclipse-platform/eclipse.platform.releng.git) which has been migrated to a new one (https://github.com/eclipse-platform/eclipse.platform.releng.aggregator.git) but the new one doesn't seem to contain the code for the older versions (< 4.29 or 4.30). I don't know if that is by design or an error during migration, but it doesn't matter now. We now have the correct version of that feature.

[eposse]

That seems to have worked (the build, I haven't tested the RCP yet). It now includes updated versions of the Eclipse Platform, RCP, e4 RCP, GMF Runtime and Apache XMLGraphics.

Now I'll move on to one of the three left:

• org.apache.ant
• org.apache.httpcomponents.httpclient
• org.freemarker

[eposse]

A status update:\

I started working on the patch for org.apache.httpcomponents.httpclient.

Patching it was easy, but I have not been able to integrate it into the build because it has two non-OSGI dependencies.

In case you might not be familiar with this, Eclipse is a platform that implements the OSGI standard, a standard for development and deploying Java packages as modules. Eclipse plugins are OSGI "bundles" which are essentially jar files with extra meta-data, including of course, dependencies. Dependencies can be other OSGI bundles or plain Java packages available in the class-path. Now, Maven doesn't know anything about OSGI bundles and in particular it doesn't know how to deal with OSGI dependencies. On the other hand, Tycho, the Maven extension we use to build, does. Unfortunately, Tycho doesn't know how to handle non-OSGI dependencies. It expects them to be in a p2 site. But in many cases, like the one I'm dealing with now, dependencies don't exist as OSGI bundles on p2 sites. And this is the situation that I'm dealing with. However, I found out about a Maven plugin called p2-maven, which allows you to get non-OSGI dependencies, turn them into OSGI bundles and make a p2 site with them which can then be used by Tycho. So I am trying to use this to integrate those non-OSGI dependencies.

[eposse]

An update:

I was finally able to integrate apache httpclient to the build. It turns out the p2-maven plugin was not necessary. Apparently Tycho added the required functionality sometime in 2020 (see Bug 567098 - pomDependencies=consider should wrap non-osgi jars . The trick was to add the non-OSGI dependencies as normal maven dependencies and add the following to the tycho target-platform-configuration:

<pomDependencies>consider</pomDependencies>
<pomDependencies>wrapAsBundle</pomDependencies>

That took a long time to find out. But it was worth learning this, as I think this is probably going to resolve the issue I was having earlier building apache ant as well.

So right now, only two more patches left:

• org.apache.ant
• org.freemarker

I'll try apache ant first to see if I can integrate it with the solution above.

[eposse]

An update:

After finishing the 2.5.0 release (Issue #481), I went back to the patches, but I ran into some unexpected problems: in order to test the new papyrus branch which bumped versions to 2.6.0 as well as the new streams/v2.5.x-maintenance branch, I had to rebuild them, but a standard maven clean build failed, presumably because my latest builds are based on Papyrus 6.5 rather than 4.8 as the older branches, and therefore a lot of newer artifacts had been added to the maven repository that are incompatible with the older builds. So I had to reset the local maven repository (~/.m2/repository) to build the versions based on Papyrus 4.8. But when I went back to the build based on Papyrus 6.5, it failed building some patches for Xtend2 and Papyrus itself. The errors are all related to Xtend2, in particular, there are a number of plugins that include Xtend2 code which is supposed to be compiled into Java at build-time. But now the generated code results in several compile-time errors. I think the the build used to work because I had probably included pre-generated code. I have been able to fix one of the Papyrus plugins, but one of the Xtend2 plugins is giving me a lot of trouble, and this one is critical, because the generated Java code includes references to the old Log4j 1. Unfortunately it's a bit of a chicken-and-egg problem: the older, pre-installed version of Xtend2 is the one generating the code, but it generates code with a reference to the old log4j, which we have removed, resulting in a compilation error on the generated Java. But I can only update the patched Xtend2 to generate the correct code. But this means that I need the the patched Xtend2 to build the patched Xtend2. I'm not yet sure how to resolve this circular build problem. I think I may have to try pre-generating code and then manually updated the pre-generated code, commit the updated generated code and disable code generation for this plugin. That's the only solution that comes to mind at least for now. I'm working on that.

[eposse]

It looks like I was able to resolve the Xtend2 build issues by committing the generated xtend files to the repo.

[eposse]

Status update:

After resolving the Xtend2 issue I started looking at Apache Ant. Unfortunately I ran into a lot of trouble, as Ant is built in a radically different way to every other plugin. So today I spent the day trying looking instead at the other pending dependency: org.freemarker. But I ran into trouble there too. It's not as bad as Ant, but I still have a compile-time issue that seems to be about conflicting dependencies. I am looking into possible ways to resolve it.

[eposse]

An update:

Building org.freemarker is proving to be a bit of a nightmare. It is extremely old, with extremely old dependencies, and it is usually built with Ant. I've encountered many problems. I've been able to solve some, but currently I'm dealing with the issue that its source code includes classes that are supposed to be compiled conditionally depending on the build configuration and which are mutually exclusive, so I'm trying to figure out, first, which is the right subset of classes for a configuration, second, figure out which configuration we need for it, and third, how to select or filter the relevant classes in tycho/maven. It's tricky, but I think I have an idea on how to do it.