eclipse-birt / birt

Eclipse BIRT™ The open source reporting and data visualization project.
http://www.eclipse.org/birt
Eclipse Public License 2.0
456 stars 391 forks source link

Derby missing from runtime #1618

Closed SteveSchafer-Innovent closed 7 months ago

SteveSchafer-Innovent commented 7 months ago
  1. clone current BIRT repo (4.16)
  2. build
  3. copy build/birt-packages/birt-runtime/target/package/birt.war into a tomcat 9.0.13 webapps dir.
  4. BIRT installation page looks good.
  5. Example 01: Integrated simple report looks good.
  6. Example 02: Integrated chart report fails because Derby isn't present.

Traceback: org.eclipse.birt.report.engine.api.EngineException: An exception occurred during processing. Please see the following message for details: Cannot open the connection for the driver: org.eclipse.birt.report.data.oda.jdbc. org.eclipse.birt.report.data.oda.jdbc.JDBCException: Failed to get connection. SQL error #1:org.apache.derby.jdbc.EmbeddedDriver ; java.sql.SQLException: org.apache.derby.jdbc.EmbeddedDriver at org.eclipse.birt.report.engine.executor.ExecutionContext.addException(ExecutionContext.java:1060) at org.eclipse.birt.report.engine.executor.ExecutionContext.addException(ExecutionContext.java:1045) at org.eclipse.birt.report.engine.executor.QueryItemExecutor.executeQuery(QueryItemExecutor.java:83) at org.eclipse.birt.report.engine.executor.ListItemExecutor.execute(ListItemExecutor.java:65) at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.execute(WrappedReportItemExecutor.java:45) at org.eclipse.birt.report.engine.internal.executor.emitter.ReportItemEmitterExecutor.execute(ReportItemEmitterExecutor.java:45) at org.eclipse.birt.report.engine.internal.executor.dup.SuppressDuplicateItemExecutor.execute(SuppressDuplicateItemExecutor.java:41) at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.execute(WrappedReportItemExecutor.java:45) at org.eclipse.birt.report.engine.internal.executor.l18n.LocalizedReportItemExecutor.execute(LocalizedReportItemExecutor.java:34) at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:62) at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:27) at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:133) at org.eclipse.birt.report.engine.layout.html.HTMLInlineStackingLM.resumeLayout(HTMLInlineStackingLM.java:97) at org.eclipse.birt.report.engine.layout.html.HTMLInlineStackingLM.layoutNodes(HTMLInlineStackingLM.java:136) at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:27) at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:133) at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:66) at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:27) at org.eclipse.birt.report.engine.layout.html.HTMLRepeatHeaderLM.layoutChildren(HTMLRepeatHeaderLM.java:54) at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:133) at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:66) at org.eclipse.birt.report.engine.layout.html.HTMLPageLM.layout(HTMLPageLM.java:92) at org.eclipse.birt.report.engine.layout.html.HTMLReportLayoutEngine.layout(HTMLReportLayoutEngine.java:97) at org.eclipse.birt.report.engine.presentation.ReportDocumentBuilder.build(ReportDocumentBuilder.java:226) at org.eclipse.birt.report.engine.api.impl.RunTask.doRun(RunTask.java:224) at org.eclipse.birt.report.engine.api.impl.RunTask.run(RunTask.java:85) at org.eclipse.birt.report.service.ReportEngineService.runReport(ReportEngineService.java:1086) at org.eclipse.birt.report.service.BirtViewerReportService.runReport(BirtViewerReportService.java:146) at org.eclipse.birt.report.service.actionhandler.BirtRunReportActionHandler.execute(BirtRunReportActionHandler.java:79) at org.eclipse.birt.report.service.actionhandler.BirtGetPageActionHandler.checkDocumentExists(BirtGetPageActionHandler.java:56) at org.eclipse.birt.report.service.actionhandler.AbstractGetPageActionHandler.prepareParameters(AbstractGetPageActionHandler.java:114) at org.eclipse.birt.report.service.actionhandler.AbstractGetPageActionHandler.execute(AbstractGetPageActionHandler.java:100) at org.eclipse.birt.report.service.actionhandler.AbstractBaseActionHandler.execute(AbstractBaseActionHandler.java:88) at org.eclipse.birt.report.soapengine.processor.AbstractBaseDocumentProcessor.executeAction(AbstractBaseDocumentProcessor.java:45) at org.eclipse.birt.report.soapengine.processor.AbstractBaseComponentProcessor.executeAction(AbstractBaseComponentProcessor.java:132) at org.eclipse.birt.report.soapengine.processor.BirtDocumentProcessor.handleGetPage(BirtDocumentProcessor.java:83) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.eclipse.birt.report.soapengine.processor.AbstractBaseComponentProcessor.process(AbstractBaseComponentProcessor.java:107) at org.eclipse.birt.report.soapengine.endpoint.BirtSoapBindingImpl.getUpdatedObjects(BirtSoapBindingImpl.java:67) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397) at org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186) at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:321) at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32) at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118) at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83) at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:420) at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281) at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:684) at org.eclipse.birt.report.servlet.BirtSoapMessageDispatcherServlet.doPost(BirtSoapMessageDispatcherServlet.java:227) at javax.servlet.http.HttpServlet.service(HttpServlet.java:660) at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:332) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.eclipse.birt.report.servlet.BirtSoapMessageDispatcherServlet.service(BirtSoapMessageDispatcherServlet.java:115) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.eclipse.birt.report.filter.ViewerFilter.doFilter(ViewerFilter.java:71) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:668) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:791) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1417) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:840) Caused by: org.eclipse.birt.report.data.adapter.api.AdapterException: An exception occurred during processing. Please see the following message for details: Cannot open the connection for the driver: org.eclipse.birt.report.data.oda.jdbc. org.eclipse.birt.report.data.oda.jdbc.JDBCException: Failed to get connection. SQL error #1:org.apache.derby.jdbc.EmbeddedDriver ; java.sql.SQLException: org.apache.derby.jdbc.EmbeddedDriver at org.eclipse.birt.report.data.adapter.impl.DataRequestSessionImpl.execute(DataRequestSessionImpl.java:607) at org.eclipse.birt.report.engine.data.dte.DteDataEngine.doExecuteQuery(DteDataEngine.java:138) at org.eclipse.birt.report.engine.data.dte.DataGenerationEngine.doExecuteQuery(DataGenerationEngine.java:74) at org.eclipse.birt.report.engine.data.dte.AbstractDataEngine.execute(AbstractDataEngine.java:254) at org.eclipse.birt.report.engine.executor.ExecutionContext.executeQuery(ExecutionContext.java:1624) at org.eclipse.birt.report.engine.executor.QueryItemExecutor.executeQuery(QueryItemExecutor.java:73) ... 82 more Caused by: org.eclipse.birt.data.engine.odaconsumer.OdaDataException: Cannot open the connection for the driver: org.eclipse.birt.report.data.oda.jdbc. org.eclipse.birt.report.data.oda.jdbc.JDBCException: Failed to get connection. SQL error #1:org.apache.derby.jdbc.EmbeddedDriver ; java.sql.SQLException: org.apache.derby.jdbc.EmbeddedDriver at org.eclipse.birt.data.engine.odaconsumer.ExceptionHandler.newException(ExceptionHandler.java:56) at org.eclipse.birt.data.engine.odaconsumer.ConnectionManager.openConnection(ConnectionManager.java:169) at org.eclipse.birt.data.engine.executor.DataSource.newConnection(DataSource.java:210) at org.eclipse.birt.data.engine.executor.DataSource.open(DataSource.java:199) at org.eclipse.birt.data.engine.impl.DataSourceRuntime.openOdiDataSource(DataSourceRuntime.java:195) at org.eclipse.birt.data.engine.impl.QueryExecutor.openDataSource(QueryExecutor.java:421) at org.eclipse.birt.data.engine.impl.QueryExecutor.prepareExecution(QueryExecutor.java:305) at org.eclipse.birt.data.engine.impl.PreparedQuery.doPrepare(PreparedQuery.java:388) at org.eclipse.birt.data.engine.impl.PreparedDataSourceQuery.produceQueryResults(PreparedDataSourceQuery.java:164) at org.eclipse.birt.data.engine.impl.PreparedDataSourceQuery.execute(PreparedDataSourceQuery.java:153) at org.eclipse.birt.data.engine.impl.PreparedOdaDSQuery.execute(PreparedOdaDSQuery.java:172) at org.eclipse.birt.report.data.adapter.impl.DataRequestSessionImpl.execute(DataRequestSessionImpl.java:594) ... 87 more Caused by: org.eclipse.birt.report.data.oda.jdbc.JDBCException: Failed to get connection. SQL error #1:org.apache.derby.jdbc.EmbeddedDriver ; java.sql.SQLException: org.apache.derby.jdbc.EmbeddedDriver at org.eclipse.birt.report.data.oda.jdbc.Connection.connectByUrl(Connection.java:226) at org.eclipse.birt.report.data.oda.jdbc.Connection.open(Connection.java:142) at org.eclipse.datatools.connectivity.oda.consumer.helper.OdaConnection.open(OdaConnection.java:250) at org.eclipse.birt.data.engine.odaconsumer.ConnectionManager.openConnection(ConnectionManager.java:160) ... 97 more Caused by: java.sql.SQLException: org.apache.derby.jdbc.EmbeddedDriver at org.eclipse.birt.report.data.oda.sampledb.SampleDBJDBCConnectionFactory.getDerbyDriver(SampleDBJDBCConnectionFactory.java:93) at org.eclipse.birt.report.data.oda.sampledb.SampleDBJDBCConnectionFactory.getConnection(SampleDBJDBCConnectionFactory.java:69) at org.eclipse.birt.report.data.oda.jdbc.JDBCDriverManager.doConnect(JDBCDriverManager.java:248) at org.eclipse.birt.report.data.oda.jdbc.JDBCDriverManager.getConnection(JDBCDriverManager.java:222) at org.eclipse.birt.report.data.oda.jdbc.Connection.connectByUrl(Connection.java:220) ... 100 more

speckyspooky commented 7 months ago

@merks Have you an idea why we lost the embedded Derby driver?

merks commented 7 months ago

You'll recall that I've repeatedly whined about the mysterious details of the various ant tasks that copy things according to various glob patterns that may or may not keep working as bundle names, bundle versions, or nesting of libraries changes. It's completely fragile and hence error prone, no one quite understands what they do, and no one notices when things go wrong, except perhaps weeks or months after the fact, because nothing tests the results.

You may recall that I've also complained that I cannot get the "runtime tests" working:

image

Though it's not clear if those are just a sniff test or actually do some deeper testing.

In any case, if there are not tests for it, and no one tests it manually, not even before a release, then it will definitely end up being broken sooner or later.

Oh well, enough whining and ranting....


Looking at the source, I wonder where something might have gone wrong? Some of the looks like unused noise. Does anything jump out at you folks that might be wrong?

image

There is the assertion in the title that derby is missing when in fact it is present:

image

But then probably "embedded Derby driver" refer to something else? Should a jar with that name exist somewhere?

image

SteveSchafer-Innovent commented 7 months ago

Derby is in fact present in some places, but not the right places. I think I've tracked it down to the ant build in birt.runtime.osgi. I'm experimenting with it now.

speckyspooky commented 7 months ago

I have tested OSGI & standard-runtime of BIRT 4.15 again and all demo-reports work(!)

Used Tomcat "9.0.85" Java 21 and our BIRT 4.15. The OS is MS Windows 11 and currently I'm not able to produce the error. All works on my side:

Screen of BIRT 4.15 OSGi with both demo reports

osgi_4 15

Screen of BIRT 4.15 Standard (w/o OSGi) with both demo reports

rt_4 15

@SteveSchafer-Innovent Now I'm finished the retest based on the war-files for both versions OSGi & standard and all works fine. I'm note able to get your error.

Screen of BIRT 4.16 OSGi with both demo reports

osgi_4 16

speckyspooky commented 7 months ago

@merks Thanks foru your help here, I know that the the buidl process is a nightmare. I invested a lot of time for testing but I don't know in full what will processed to get a final version.

So I tested 6 versions now manually to get the results and it looks good again according to the milestone tests.

@SteveSchafer-Innovent Please note the information of the start-page of birt "Viewer & Engine version" is 4.14.0 but it is really 4.15. Additional on it the magical version-jar from Ed set now the version number correctly - see my last screen. (I also double checked the source and it looks good.)

merks commented 7 months ago

@speckyspooky

I'm not pointing fingers anywhere specifically. You're almost like a saint what you all do for the good of the project technically! 👼 I just try to be a good janitor, which is kind of challenging...

SteveSchafer-Innovent commented 7 months ago

I downloaded 4.15 runtime and I get the same results as you do. However it fails when I clone the repo and build 4.16. I need to be able to do this in order to test other changes I'm making to BIRT. So the difference is 1) it's the current master branch (4.16), or 2) I'm building it, or 3) I'm building it locally on linux vs using the build result from Jenkins or Github or wherever it comes from.

Is there a reason I shouldn't be able to build it locally and use the .war file that is produced by that build?

merks commented 7 months ago

Perhaps the devil is in the details of what "building it" really means. You're actually doing a local Maven/Tycho build?

SteveSchafer-Innovent commented 7 months ago

mvn clean verify -DskipTests=true from within the top-level directory.

Looking through the build log I see a lot of warnings like this:

[WARNING] The following locally built units have been used to resolve project dependencies: [WARNING] org.apache.derby/10.15.2.v202206102045

Do you think that has any bearning?

SteveSchafer-Innovent commented 7 months ago

In my previous comment, that warning refers to a directory, not a jar file. In fact i find that directory in many of the places that should have a jar file. I think the jar file should be there because the ant scripts copy that directory, for example, from birt.runtime.osgi to birt.runtime. I think i'll try deleting my .m2 repository folder and see if that helps.

merks commented 7 months ago

That's not the version I would expect from what's in my target platform:

image

Nor the one I see included in the war:

image

You're not on a fork that's different from master? Often people clone their own forks and then forget to sync their master branch....

You might consider to create a fresh installation with a fresh clone that directly using the original repository URI, not a fork:

https://github.com/eclipse-birt/birt/blob/master/CONTRIBUTING.md

SteveSchafer-Innovent commented 7 months ago

I'm not using a fork. In fact that was the first thing I tried and then switched to a direct clone when that didn't work. I'm trying it now with a fresh .m2 folder to see if that helps.

merks commented 7 months ago

This is what I see when I do Ctrl-Shift-R and try org.apache.derby:

image

SteveSchafer-Innovent commented 7 months ago

After deleting .m2 the build gets a lot of errors in the viewer. The first one is this:

[ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:4.0.7:compile (default-compile) on project org.eclipse.birt.report.viewer: Compilation failure: Compilation failure: [ERROR] /disk1/home/innovent/git/birt-clone/birt/viewer/org.eclipse.birt.report.viewer/birt/WEB-INF/classes/org/eclipse/birt/report/service/actionhandler/AbstractBaseActionHandler.java:[20] [ERROR] import javax.xml.namespace.QName; [ERROR] ^^^^^^^^^^^^^^^^^^^ [ERROR] The package javax.xml.namespace is accessible from more than one module: , java.xml

Java version is 17 (openjdk).

speckyspooky commented 7 months ago

The same issue was documented last year November - here #1492 & #1493

SteveSchafer-Innovent commented 7 months ago

So Ed merged a change last November that fixed it but now it's back? Is regenerating the target platform something I need to do, and if so, can someone tell me how to do it?

speckyspooky commented 7 months ago

This is a little bit more difficult because the plattform update solved the topic. I have currently the same problem with a new PR.

But I'm not able to say what we can do to solve the problem.

merks commented 7 months ago

Sorry it’s almost 7:00pm for me so it’s past time to help. I assume you have maven 3.9.x. It’s difficult to imagine how the .m2 cache can cause this build failure.

It’s so late I missed to hit the comment button.

SteveSchafer-Innovent commented 7 months ago

I just restored my original .m2 directory and now it builds without error. I think Arthur C. Clark said "Any sufficiently complex system is indistinguishable from magic", or something like that.

Of course now the derby problem is back.

I tried comparing the .m2 directories but they have way too many differences for me to find anything useful.

SteveSchafer-Innovent commented 7 months ago

I added a line to build.xml in birt.runtime.osgi to force it to copy org.apache.derby*.jar to birt.runtime and was elated for a second when I saw that the jar file did indeed get copied to the WEB-INF/lib folder, and then I ran the sample chart report and it failed the same way as before. Then I took a closer look at that jar file and found that it doesn't contain any classes! It's basically an empty shell. I guess this fits with these hundreds of warnings I'm getting:

[WARNING] The following locally built units have been used to resolve project dependencies: [WARNING] org.apache.derby/10.15.2.v202206102045

Apparently it's just making stuff up. Ugh.

merks commented 7 months ago

I did try to update to a newer derby in the past and eventually I discovered that it's just kind of empty, so I backed it out pending more time to figure out how to package a new one properly. But I still don't understand why you are seeing/using that version. It suggests your .target is not the same as my .target on master. Or some funky .m2 cache is making a difference. Note I have committed an m2e launcher to do the build:

image

I would suggest using that to do the build so that you see what I see. In there you see there are some properties about the base line so perhaps in your build there is some baseline replacement kicking in result in stuff that look older. It's also possible to change the user.home value of this launcher so a different .m2 cache is used (because it's always located in the home folder).


The QName problem is annoying and stems from the fact that indeed there are multiple things that export this package, including the JRE itself:

image

I've been trying to stamp out these older sources, but getting everyone to move forward is a challenge at best, and BIRT is effectively downstream from everyone. My sense is that Tycho build is using a different classpath than what I see in the IDE, so I will find a way to force it not to use older bundles that are causing this problem. It looks like forcing the use of the newer javax.xml.rpc-api;bundle-version="1.1.4" does the trick so I will create a PR for that to verify it fixes the build.

SteveSchafer-Innovent commented 7 months ago

@merks I've been avoiding eclipse because there are currently many errors. After I do a clean all projects, there are 4114 errors. Then after doing a maven update project on all projects there are still 4114 errors. I created the eclipse environment by using CREATE DEVELOPMENT ENVIRONMENT on the Github BIRT main readme. It's based on my fork but my fork is currently synced and my local is up to date.

merks commented 7 months ago

Don't do a maven update. Only use the setup . Do it again cleanly from scratch. If you do some other things some other way, there is a limit on what I can do to help because I can't replicate what you are doing.

SteveSchafer-Innovent commented 7 months ago

Okay, I was able to re-install. Possibly the original problem was that I was unaware that it was going to clone my fork and my fork was probably seriously out of date at the time. Anyway, I now have 2 errors, which are

Failed to execute mojo org.apache.maven.plugins:maven-antrun-plugin:3.1.0:run {execution: jar-pre-compile-jsp} ... Failed to execute mojo org.apache.maven.plugins:maven-antrun-plugin:3.1.0:run {execution: javadoc} ...

Tracebacks attached: eclipse-errors.txt

Update: I was able to run BIRT Tycho Build in eclipse and it succeeded. The derby jar files are all still 10.15.2 and are all still empty.

Update 2: I deleted the .m2 folder and reran BIRT Tycho Build and it succeeded! The derby jar files are now 10.11.1. The sample chart report works!

speckyspooky commented 7 months ago

The 2 mentioned errors are also listed on my eclipse environment. But the both haven't currently any impact of the project. The designer, viewer and runtime engine works fine after the build also with the 2 topics.

merks commented 7 months ago

Yes delete annoying maven errors. 😱

merks commented 7 months ago

FYI, if you do a Tycho build in the IDE (i.e., in the clone used by the IDE), it will mess up the state of the IDE. You have to do a Project -> Clean -> Clean all projects to rebuilds the IDE state.

That build produces a small number of "Maven Build Problem" errors.

image

Select them and delete them. I'm never so happy when folks at the m2e nature to plugin projects:

image

Also when folks decide the IDE should compile classes to the same location as the Tycho build such that one will stomp the other:

image

But such is the state of the project.

I don't expect you actually need to do a Tycho build in order to debug problems...

SteveSchafer-Innovent commented 7 months ago

I did do a command-line build after doing the Tycho build and it worked. But it appears I need to do a Tycho build first to get everything into the proper state for the command line build to work. Comparing birt-master5 (where I did the command-line build) to birt-master6 (where I did the IDE build), I see that all the .classpath files are different and not much else.

I seem to remember making modifications to org.eclipse.birt.target.target in order to get it to use various different orbit repositories, but now that file seems to be generated. So I'm wondering, where in all these files do we specify that we want the org.apache.derby jar file?

merks commented 7 months ago

You're asking how does the generator know to generate this line?

https://github.com/eclipse-birt/birt/blob/0d28caca3f9806a6e0a16c24f99977b36a1afb74/build/org.eclipse.birt.target/org.eclipse.birt.target.target#L57

The IDE uses a Modular Targlet Task like this:

image

That implementation induces a p2 repository from the git clone source and uses the p2 planner do resolve all the requirements of all the source bundles, features, and products against the available installable units in the listed of repositories. It uses that solution to populate the target platform in the IDE and uses the TargetDefinitionGenerator annotation to also generate a *.target file that captures that solution as a side effect.

This makes it easier to evolve the target platform as new versions of dependencies become available.

The Orbit project generally finds new versions and makes them available:

https://github.com/eclipse-orbit

But derby is a complicated beast, and I've not had time to properly investigate how to wrap/bundle a new version of that after my previous aborted attempt...

SteveSchafer-Innovent commented 7 months ago

That's a screen I've not seen before and I'm not sure how to get to it.

merks commented 7 months ago

Help -> Perform Setup Tasks. Or the left of these buttons:

image

The serialization is here:

https://github.com/eclipse-birt/birt/blob/0d28caca3f9806a6e0a16c24f99977b36a1afb74/build/org.eclipse.birt.releng/BIRT.setup#L401-L504

Best never to edit that textual but with the Setup Editor.

So generally one doesn't modify the target but rather change the dependencies in a MANIFEST.MF or a feature.xml and run the setup tasks again to regenerate the new target.

merks commented 7 months ago

@SteveSchafer-Innovent

Can we close this issue as resolved? I hope you are successfully making progress...

SteveSchafer-Innovent commented 7 months ago

Thanks for all your help Ed and Thomas. I would prefer that one could clone the BIRT master and then run the build from the command line without having to do it in Eclipse first. I'll continue to see if I can do that, but for now I think we can close this.

merks commented 7 months ago

Both the GitHub action and the Jenkins build do exactly what you ask/describe.

There is absolutely no magic in the ide. It’s just that the oomph setup reproduces exactly the same for you as for me. But which maven version you use with which version of Java and with which parameters for maven I can’t be sure exactly what you use. But the GitHub action, the Jenkins build, and the m2e launcher in the ide all do work and do what you want. So I can hope that 3 working examples will light the way.