search

eclipse-tycho / tycho

Tycho project repository (tycho)
https://tycho.eclipseprojects.io
Eclipse Public License 2.0
166 stars 189 forks source link

A lot of bundles suddenly unpacked in product #3346

Closed marschall closed 3 months ago

marschall commented 8 months ago

Tycho Version: 4.0.4

We use Tycho to build an Eclipse product with materialize-products. We recently switched our target platform from 4.28 / 2023-06 to 4.30 / 2023-12 and suddenly most bundles are unpacked. Most notably the launcher.jar is suddenly unpacked and therefore the launcher no longer works.

Bundles now unpacked

com.sun.jna_5.13.0.v20230812-1000/
com.telekurs.pass.rcp.ssi_94.0.0.202401091451/
org.apache.felix.gogo.command_1.1.2/
org.apache.felix.gogo.runtime_1.1.6/
org.apache.felix.gogo.shell_1.1.4/
org.apache.xmlgraphics_2.9.0.v20230916-1600/
org.eclipse.core.commands_3.11.200.v20231108-1058/
org.eclipse.core.contenttype_3.9.200.v20230914-0751/
org.eclipse.core.databinding.beans_1.10.100.v20230708-0916/
org.eclipse.core.databinding.observable_1.13.100.v20230708-0916/
org.eclipse.core.databinding.property_1.10.100.v20230708-0916/
org.eclipse.core.databinding_1.13.100.v20230708-0916/
org.eclipse.core.expressions_3.9.200.v20230921-0857/
org.eclipse.core.filesystem_1.10.200.v20231102-0934/
org.eclipse.core.jobs_3.15.100.v20230930-1207/
org.eclipse.core.net_1.5.200.v20231106-1240/
org.eclipse.core.resources_3.20.0.v20231102-0934/
org.eclipse.core.runtime_3.30.0.v20231102-0719/
org.eclipse.e4.core.commands_1.1.200.v20231108-1058/
org.eclipse.e4.core.contexts_1.12.400.v20231103-2012/
org.eclipse.e4.core.di.annotations_1.8.200.v20231103-2012/
org.eclipse.e4.core.di.extensions.supplier_0.17.300.v20231028-1632/
org.eclipse.e4.core.di.extensions_0.18.100.v20231103-2012/
org.eclipse.e4.core.di_1.9.200.v20231105-0754/
org.eclipse.e4.core.services_2.4.200.v20231103-2012/
org.eclipse.e4.emf.xpath_0.4.100.v20230801-1255/
org.eclipse.e4.ui.bindings_0.14.200.v20231030-2045/
org.eclipse.e4.ui.css.core_0.14.200.v20231025-0632/
org.eclipse.e4.ui.css.swt.theme_0.14.200.v20230913-1724/
org.eclipse.e4.ui.css.swt_0.15.200.v20231025-0632/
org.eclipse.e4.ui.di_1.5.200.v20231030-2045/
org.eclipse.e4.ui.dialogs_1.4.100.v20230727-0604/
org.eclipse.e4.ui.model.workbench_2.4.100.v20230801-1255/
org.eclipse.e4.ui.progress_0.4.200.v20231030-2045/
org.eclipse.e4.ui.services_1.6.200.v20231030-2045/
org.eclipse.e4.ui.swt.win32_1.2.100.v20231030-2045/
org.eclipse.e4.ui.widgets_1.4.0.v20230317-0802/
org.eclipse.e4.ui.workbench.addons.swt_1.5.200.v20231106-1600/
org.eclipse.e4.ui.workbench.renderers.swt_0.16.200.v20231030-2045/
org.eclipse.e4.ui.workbench.swt_0.17.200.v20231030-2045/
org.eclipse.e4.ui.workbench_1.15.200.v20231030-2045/
org.eclipse.e4.ui.workbench3_0.17.200.v20231025-0632/
org.eclipse.equinox.app_1.6.400.v20231103-0807/
org.eclipse.equinox.bidi_1.4.400.v20231106-0858/
org.eclipse.equinox.common_3.18.200.v20231106-1826/
org.eclipse.equinox.console_1.4.600.v20231106-0859/
org.eclipse.equinox.event_1.6.300.v20231012-1251/
org.eclipse.equinox.http.jetty_3.9.0.v20231104-1114/
org.eclipse.equinox.http.registry_1.3.400.v20231012-1210/
org.eclipse.equinox.http.servlet_1.8.0.v20231018-1316/
org.eclipse.equinox.jsp.jasper.registry_1.2.200.v20231012-1239/
org.eclipse.equinox.jsp.jasper_1.1.800.v20231012-1238/
org.eclipse.equinox.launcher.win32.win32.x86_64_1.2.800.v20231003-1442/
org.eclipse.equinox.preferences_3.10.400.v20231102-2218/
org.eclipse.equinox.registry_3.11.400.v20231102-2218/
org.eclipse.equinox.security_1.4.100.v20231012-1825/
org.eclipse.equinox.simpleconfigurator_1.5.0.v20230919-1457/
org.eclipse.help.base_4.4.200.v20231201-0110/
org.eclipse.help.ui_4.6.100.v20231106-1240/
org.eclipse.help.webapp_3.11.200.v20231108-0716/
org.eclipse.help_3.10.200.v20231106-1240/
org.eclipse.jdt.core.compiler.batch_3.36.0.v20231114-0937/
org.eclipse.jdt.core_3.36.0.v20231115-1055/
org.eclipse.jface.databinding_1.15.100.v20230708-0916/
org.eclipse.jface.notifications_0.7.0.v20231017-1913/
org.eclipse.jface.text_3.24.200.v20231024-0828/
org.eclipse.jface_3.32.0.v20231107-0834/
org.eclipse.osgi.compatibility.state_1.2.900.v20231106-0859/
org.eclipse.osgi.services_3.11.200.v20231106-0901/
org.eclipse.osgi.util_3.7.300.v20231104-1118/
org.eclipse.osgi_3.18.600.v20231110-1900/
org.eclipse.rcp_4.30.0.v20231201-0110/
org.eclipse.swt.win32.win32.x86_64_3.124.200.v20231113-1355/
org.eclipse.swt_3.124.200.v20231113-1355/
org.eclipse.text_3.13.100.v20230801-1334/
org.eclipse.ui.forms_3.13.100.v20231102-1012/
org.eclipse.ui.workbench_3.131.0.v20231201-0512/
org.eclipse.ui_3.205.0.v20230922-1312/
org.eclipse.update.configurator_3.5.200.v20231102-0934/
org.eclipse.urischeme_1.3.100.v20230727-0604/
org.osgi.service.cm_1.6.1.202109301733/
org.osgi.service.component_1.5.1.202212101352/
org.osgi.service.device_1.1.1.202109301733/
org.osgi.service.event_1.4.1.202109301733/
org.osgi.service.metatype_1.4.1.202109301733/
org.osgi.service.prefs_1.1.2.202109301733/
org.osgi.service.provisioning_1.2.0.201505202024/
org.osgi.service.upnp_1.2.1.202109301733/
org.osgi.service.useradmin_1.1.1.202109301733/
org.osgi.service.wireadmin_1.0.2.202109301733/
org.osgi.util.function_1.2.0.202109301733/
org.osgi.util.measurement_1.0.2.201802012109/
org.osgi.util.position_1.0.1.201505202026/
org.osgi.util.promise_1.3.0.202212101352/
org.osgi.util.xml_1.0.2.202109301733/

Bundles still packaged

com.ibm.icu_74.1.0.jar
com.sun.el.javax.el_3.0.4.jar
com.sun.jna.platform_5.13.0.jar
jakarta.annotation-api_2.1.1.jar
jakarta.inject.jakarta.inject-api_2.0.1.jar
javax.annotation_1.3.5.v20200909-1856.jar
org.apache.aries.spifly.dynamic.bundle_1.3.7.jar
org.apache.batik.constants_1.17.0.v20231009-1000.jar
org.apache.batik.css_1.17.0.v20231009-1000.jar
org.apache.batik.i18n_1.17.0.v20231009-1000.jar
org.apache.batik.util_1.17.0.v20231009-1000.jar
org.apache.commons.cli_1.6.0.jar
org.apache.commons.collections_3.2.2.jar
org.apache.commons.commons-beanutils_1.9.4.jar
org.apache.commons.commons-codec_1.16.0.jar
org.apache.commons.commons-collections4_4.4.0.jar
org.apache.commons.commons-io_2.15.0.jar
org.apache.commons.jxpath_1.3.0.jar
org.apache.commons.lang3_3.13.0.jar
org.apache.commons.logging_1.2.0.jar
org.apache.felix.scr_2.2.6.jar
org.apache.logging.log4j.api_2.21.1.jar
org.apache.lucene.analysis-common_9.8.0.v20230929-1030.jar
org.apache.lucene.analysis-smartcn_9.8.0.v20230929-1030.jar
org.apache.lucene.core_9.8.0.v20230929-1030.jar
org.eclipse.emf.common_2.29.0.v20230916-0637.jar
org.eclipse.emf.ecore.change_2.15.0.v20230211-1150.jar
org.eclipse.emf.ecore.xmi_2.36.0.v20231002-1156.jar
org.eclipse.emf.ecore_2.35.0.v20230829-0934.jar
org.eclipse.equinox.launcher_1.6.600.v20231106-1826.jar
org.eclipse.jetty.ee8.security_12.0.3.jar
org.eclipse.jetty.ee8.server_12.0.3.jar
org.eclipse.jetty.ee8.servlet_12.0.3.jar
org.eclipse.jetty.http_12.0.3.jar
org.eclipse.jetty.io_12.0.3.jar
org.eclipse.jetty.security_12.0.3.jar
org.eclipse.jetty.server_12.0.3.jar
org.eclipse.jetty.servlet-api_4.0.6.jar
org.eclipse.jetty.session_12.0.3.jar
org.eclipse.jetty.util_12.0.3.jar
org.eclipse.orbit.xml-apis-ext_1.0.0.v20230923-0644.jar
org.jboss.logging.jboss-logging_3.5.3.Final.jar
org.jdom_1.1.3.v20230812-1600.jar
org.mortbay.jasper.apache-jsp_9.0.52.jar
org.objectweb.asm.commons_9.6.0.jar
org.objectweb.asm.tree.analysis_9.6.0.jar
org.objectweb.asm.tree_9.6.0.jar
org.objectweb.asm.util_9.6.0.jar
org.objectweb.asm_9.6.0.jar
slf4j.api_2.0.9.jar
akurtakov commented 8 months ago

Seems relate to https://eclipse.dev/eclipse/news/4.30/pde.php#unnecessary-attributes-removal but I can't understand why no one else complained yet.

marschall commented 8 months ago

Seems relate to https://eclipse.dev/eclipse/news/4.30/pde.php#unnecessary-attributes-removal but I can't understand why no one else complained yet.

Possible but my understanding is Tycho ignores unpack in feature.xml and instead relies on Eclipse-BundleShape see #1035

laeubi commented 8 months ago

@marschall can you please provide an example at best as an integration-test to demonstrate the issue?

You also wrote that

We recently switched our target platform from 4.28 / 2023-06 to 4.30 / 2023-12 and suddenly most bundles are unpacked.

so I assume Tycho version stays constant, can you try out the current 4.0.5-SNAPSHOT build if the problem persist?

marschall commented 8 months ago

@marschall can you please provide an example at best as an integration-test to demonstrate the issue?

I'll see what I can do. I may take some time.

so I assume Tycho version stays constant

Actually we updated from 3.0.4 to 4.0.4 but it also happens with 3.0.4.

can you try out the current 4.0.5-SNAPSHOT build if the problem persist?

Same problem.

maxrabbi commented 6 months ago

@marschall I can confirm I verified the same problem. I was trying to move our product platform to the latest Eclipse 4.30 (from 4.29). But I stumbled upon this. What I can say is that it seems not Tycho dependent, meaning that the error occurred in both 3.0.5 (original one) and 4.0.6. I also gave it a try packaging the old product based on 4.29 using Tycho 4.0.6. No problem verified.

akurtakov commented 6 months ago

This is most likely caused by https://github.com/eclipse-equinox/p2/commit/89f12f7eaa884a627dcf9d30d48b878933ea5466

laeubi commented 6 months ago

Also see here:

Fitzcarraldo75 commented 5 months ago

Unfortunately that does not solve the unpack problem for us. We still face that issue with 4.31 and 4.32.M1.

When comparing the content.xml created by tycho-p2-repository:4.0.7:assemble-repository using 4.32.M1 with the file created when using 4.29 we see that difference in the touchpoint data:

`

true Bundle-SymbolicName: org.eclipse.equinox.launcher;singleton:=true Bundle-Version: 1.6.800.v20240330-1250 ` vs ` Bundle-SymbolicName: org.eclipse.equinox.launcher;singleton:=true Bundle-Version: 1.6.500.v20230717-2134 ` If we remove the ` true ` from content.xml before tycho-p2-director:4.0.7:materialize-products runs it works as desired for the equinox.launcher. No unpack.
maxrabbi commented 5 months ago

Hi, I thought I already reported some weeks ago that the situation did not change for me either when I tried to work with the latest 4.31. So @Fitzcarraldo75 did you use this trick to produce the correct output like it was before (in 4.29) ?

laeubi commented 5 months ago

@Fitzcarraldo75 @maxrabbi can you provide an integration-test to demonstrate the issue?

Fitzcarraldo75 commented 3 months ago

Hi there

I finally had time to look into our issue closely. It turned out that it was an issue in our infrastructure. To be more precise we upgraded to the target platform 4.31 which obviously required in our case as well a Tycho upgrade. We did that too and upgraded to Tycho 4.0.7. So far so good. We did that on a feature branch. But we also have a pipeline for building and uploading a p2 repository tailored to our application. However that pipeline runs on our "mainline" still using an old Tycho version - namely 3.0.4. That led to "zipped = true" touchpoint entries in the content.jar of the p2 repo. That was our issue. Sorry for the confusion!

Summary: we built a p2 repo out of the 4.31 platform using an old tycho version. That caused the "unpack issue".

@maxrabbi Maybe you did something similar?

marschall commented 3 months ago

I consider this fixed now.