Publish Maven Build To Central

[pipebaum]

Currently, there is no automated feature to publish the BIRT-Runtime to Maven. Wouldn't it be nice to fix this.

[SteveSchafer-Innovent]

What needs to be published is in build/birt-packages/birt-runtime/target/package/ReportEngine/lib and is created by an ant script during the build process. I may be able to publish that the same way I published com.innoventsolutions.birt.runtime. It would be nice if it was an automatic part of the birt build.

[AlexSchuetz]

It appears the link doesn't really work. It just takes you to sonatype but doesn't expand the browser tree to org/eclipse/birt, but you can still manually browse to org/eclipse/birt.

But, in the not-so-great-news department, it appears that what got published is the entirety of birt but not the birt runtime pojo jars. The pojo jars are there, but there inside the birt-runtime zip which is useless from a maven standpoint. I believe one could still utilize the repository for the runtime but it would probably need to be an osgi configuration rather than a pojo configuration. In order to have the pojo jars be mavenized I think we are going to create a new project and publish it to org.eclipse.birt.runtime, as has evidently been done in the past, up to version 4.6.0 and as I did a couple of years ago as org.innoventsolutions.birt.runtime.

I like having the whole modules in the repo and I would regret only having the runtime jar! Explanation: The runtime also includes classes that do not belong to birt but to other dependencies like google, apache or logback, which is more than just inconvenient if you want to control dependency versions. Moreover it is unexpected due to naming conventions.

Unfortunately the dependency information seems to be missing. Probably since the dependencies are not managed by maven. So one big improvement would be to manage dependencies with maven in the poms. This way transitive dependencies would be installed automatically using maven and could be controlled using exclusions. And version conflicts could be checked and resolved (i.e. maven-enforcer-plugin).

Unfortunately I don't know anything about eclipse-plugins and tycho. If someone can assist I'd be glad to work on this. But maybe that's a new ticket.

[SteveSchafer-Innovent]

I agree it's probably a good thing to have all of birt in maven but we may need to have the runtime part in addition to that. In the runtime, all the necessary classes get re-packaged into a single jar file which can be added to a project as a simple jar without having to deal with OSGi. Yes there are a lot of dependencies to both eclipse jars and non-eclipse jars. In the runtime distribution, all those jars are included and it's up to you to include them in your classpath. With maven, ideally, we could declare them as dependencies.

[AlexSchuetz]

Having a jar that includes all dependencies can be very helpful for projects that do not use maven. However if an artifact is published to the maven repository, it should also behave like a maven artifact and use the maven dependency resolution.

I think a mixed approach with some modules being pom-first on one side and the eclipse-plugins on the other would be a good idea.

Having to exclude dependencies of other frameworks or libraries like it is with the 4.8 release that you published over innoventsolutions is not the maven way. (Although I am very thankful for this artifact. Otherwise I wouldn't have considered BIRT at all.) As a developer using maven I want to be able to have a look at my projects dependency-tree. Having additional dependencies sneaked in, that may produce a runtime-error due to conflicts with my explicitly added dependencies is very inconvenient. Even more if you use tools like the maven-enforcer-plugins to detect library conflicts at build time. These tools can not detect these hidden conflicts and I wouldn't expect running accross runtime-errors caused by classloader-problems.

This is a real drawback and will influence the acceptance of the library.

Post explaining the problems of fat-jar libraries

[AlexSchuetz]

Unfortunately the following dependencies are not published as well and probably some more, that I am not aware of:

org.eclipse.datatools.connectivity_1.14.102.201911250848.jar org.eclipse.datatools.connectivity.oda_3.6.101.201811012051.jar org.eclipse.datatools.connectivity.oda.consumer_3.4.101.201811012051

[SteveSchafer-Innovent]

@AlexSchuetz since you are working on publishing the runtime and you want to explore new ways of doing that, I wanted to suggest that we do that as a part of 4.10 while I work on publishing 4.9 runtime in the traditional way, since some of Innovent's clients are wanting to get 4.9 mavenized as soon as possible. What do you think?

[AlexSchuetz]

@SteveSchafer-Innovent That's a good idea. Since version 4.9.0 is already published running the publish job won't work for 4.9.0 anyways (since republishing is prohibited).

I will work on publishing 4.9.1 on maven then. Do you want to integrate the maven branch into 4.10.0 and master already?

[SteveSchafer-Innovent]

@AlexSchuetz I think we should publish the runtime to org.eclipse.birt.runtime 4.9.0. We can use the same version because it's a different artifact id. This is also the artifact id that has been used in the past for publishing the runtime. My interest is getting this published as soon as possible.

As for 4.10.0, we can look at publishing the snapshot version of org.eclipse.birt 4.10.0. In parallel to that we can also publish the snapshot version of org.eclipse.birt.runtime 4.10.0 and experiment with different ways of packaging the artifacts. So this is what I see as the publishing tasks:

• org.eclipse.birt.runtime 4.9.0 - asap, artifacts packaged like the runtime zip, not snapshots (alex)
• org.eclipse.birt 4.10.0 - snapshots (steve)
• org.eclipse.birt.runtime 4.10.0 - snapshots, try new artifact packaging (alex and/or steve)

Does that sound good to you?

[eduardogmisiuk]

Is there any progress in this Maven build? I tried to use the 4.9.0 Maven Central build, but as said before, the org.eclipse.datatools.connectivity is not there, so I can't import it as a dependency in my pom.

[AlexSchuetz]

I am working my way through the project to provide a runtime package that will include the missing dependencies.

[AlexSchuetz]

Little update: I am working my way through the dependencies right now, trying to determine which dependency needs to be packaged into the runtime library (i.e. org.eclipse.datatools, ...) and which ones should be referenced as dependency.

Afterwards I will examine the runtime library itself and strip away things that could be referenced as a dependency (i.e. slf4j, google, apache-commons, ...).

There are still some problems ahead, though: Right now I wonder where the org.eclipse.datatool-jars originate from. I think they are downloaded somewhere, but I can't find the spot.

[AlexSchuetz]

ld publish the runtime to org.eclipse.birt.runtime 4.9.0. We can use the same version because it's a different artifact id. This is also the artifact id that has been used in the past for publishing the runtime. My interest is getting this published as soon as possible.

@SteveSchafer-Innovent Sorry for the delay. The tasks are sound. Regarding publishing the runtime for 4.9.0:

• I probably need someone with publishing rights. But maybe we can also build and publish a patch 3.9.1 completely including the runtime
• since there are some classpath problems using it as is, I am working on a condensed runtime with dependencies. The only classes that I will strip away are the ones that are already available through maven central. That's more or less what I did manually for using BIRT in a maven project.

[eduardogmisiuk]

@AlexSchuetz you can use the datatools from the runtime in eclipse downloads page, I think: https://download.eclipse.org/birt/downloads/drops/R-R1-4.9.0-202203161719/

[JacquesLeRoux]

If it's of any help, at Apache OFBiz we are interested in org.eclipse.birt.runtime:viewservlets, for security reason...

[AlexSchuetz]

If it's of any help, at Apache OFBiz we are interested in org.eclipse.birt.runtime:viewservlets, for security reason...

Good to know. You can already download the runtime.zip via maven AFAIK. The viewservlets.jar is inside. But probably that's not the desired easiest workflow. Can you give some hints about the intended use-case or what you expect exactly?

[AlexSchuetz]

@AlexSchuetz you can use the datatools from the runtime in eclipse downloads page, I think: https://download.eclipse.org/birt/downloads/drops/R-R1-4.9.0-202203161719/

That's not what I meant. I am looking for the right spot in the build process of birt, that downloads the datatools for further usage. But thank's anyway.

[wimjongman]

Alexander, the datatools are pulled from Eclipse [1]. All our target repositories are defined in the target file which is here [2]

[1] https://download.eclipse.org/datatools/updates/1.14.200-SNAPSHOT/repository/ [2] https://github.com/eclipse/birt/blob/master/build/org.eclipse.birt.target/org.eclipse.birt.target.target

[JacquesLeRoux]

Hi @AlexSchuetz,

We use Birt as a plugin component in OFBiz: https://demo-trunk.ofbiz.apache.org/birt (credential admin/ofbiz)

We use Gradle for our build. So far we use this build.gradle content for this plugin:

dependencies {
    pluginLibsCompile('org.eclipse.birt.runtime:viewservlets:4.5.0') {
        exclude group: 'org.eclipse.birt.runtime.3_7_1', module: 'derby'
        exclude group: 'org.eclipse.birt.runtime.3_7_1', module: 'org.apache.batik.pdf'
    }
}

HTH

[wimjongman]

There is some progress in Eclipse on maven:

https://www.eclipse.org/lists/eclipse-dev/msg12041.html

[SteveSchafer-Innovent]

I've taken this over from Alex because he's been having difficulties building and is also tied up with work issues. I'll take a look at the eclipse-dev message and see if it helps. Meanwhile I have a question:

I am attempting to locate the jars included in the runtime ReportEngine/lib folder in maven central, and I can locate many of them but for some of the I find that the jar in maven central is identical to the one in ReportEngine/lib with the sole exception that the manifest in ReportEngine/lib contains SHA-256-Digest keys whereas the one in maven central does not. They both contain the same SHA1-Digest keys. My question is: does this matter? Can I use these dependencies from maven central?

[wimjongman]

You can take the ones from Maven because that is the source.

[SteveSchafer-Innovent]

I've searched for every runtime jar in mvnrepository.com and found a large number of them. When I've found an artifact of the same name and version I downloaded the jar file, unzipped it, and compared it to the runtime jar. Whenever the classes are identical I included it as a dependency, even though the manifest sometimes differed in SHA signatures or other ways that I considered inconsequential. In many instances, the artifact is located in some repository other than central. Now I need to create a snapshot in sonatype. This is going to involve creating a new project in github in order to use the org.eclipse group id.

[wimjongman]

Sounds like progress, Steve. Thank you.

[AlexSchuetz]

In this pom you may find what I gathered until now.

Am Di., 5. Juli 2022 um 17:10 Uhr schrieb Wim Jongman < @.***>:

Sounds like progress, Steve. Thank you.

— Reply to this email directly, view it on GitHub https://github.com/eclipse/birt/issues/625#issuecomment-1175174875, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACA4OILE2FK4OVHFB54JAKTVSRF7NANCNFSM4ZYBQNVA . You are receiving this because you were mentioned.Message ID: @.***>

[AlexSchuetz]

Sorry. Forgot the link.

https://github.com/AlexSchuetz/birt/tree/BIRT_4.9.0_maven2/build/birt-packages/birt-runtime-maven

Am Di., 5. Juli 2022 um 19:37 Uhr schrieb Alexander Schütz < @.***>:

In this pom you may find what I gathered until now.

Am Di., 5. Juli 2022 um 17:10 Uhr schrieb Wim Jongman < @.***>:

Sounds like progress, Steve. Thank you.

— Reply to this email directly, view it on GitHub https://github.com/eclipse/birt/issues/625#issuecomment-1175174875, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACA4OILE2FK4OVHFB54JAKTVSRF7NANCNFSM4ZYBQNVA . You are receiving this because you were mentioned.Message ID: @.***>

[SteveSchafer-Innovent]

Ugh. The 4.9.0 branch is failing to build. My plan was to create a CI task that would handle the snapshot deployment with post-build scripting. The deployment needs to happen in the CI build in order to use the org.eclipse group id. The build works locally and the code hasn't changed since it built successfully back in April. I'm going to be tied up with work for a while but I'll get back to this when I can.

https://ci.eclipse.org/birt/job/birt_4.9.0_test/

@AlexSchuetz thanks for the link.

[wimjongman]

The issue is that dependencies have moved. If you want to build the exact same BIRT then I guess you have to point the target platform to the BIRT 4.9 repository.

[dee42]

is there anything I can do to help here, to unblock this?

[CrCliff]

Hi all! Is there any chance this effort could be revived?

[wimjongman]

Sure! Just dive in. Thanks for helping!

[DavidPerezIngeniero]

We can find 4.9.0 in https://mvnrepository.com/artifact/org.eclipse.birt, but what about 4.10?

There are over 100 dependencies (*.jar) in Birt runtime, and I must avoid clashes of other libraries I'm using. A maintenance nightmare.

[marangelologic]

Hi everyone, how can I have the latest jar for 4.13 version? is there a way I can package it from master? thanks

[siepkes]

I created a workaround where I upload the artifacts of the BIRT runtime ZIP and its dependencies to our local Maven repo. It might be of use to you. I've put the script here: https://gist.github.com/siepkes/00f0c410933271b4f4e0d5eab18d7469 . You can use it to upload the deps to your own repository.

[marangelologic]

@siepkes , thanks will try :)

[horosin]

@siepkes Thanks, that's what I plan to do at the moment.

Package maintainers - can we help you somehow with releasing new version to Maven? It's a big blocker, the last version is from years ago :'(

[wimjongman]

Yes we can use help!!

We have a maven based build in jenkins. https://ci.eclipse.org/birt

[horosin]

@wimjongman Great, I'll have a look this week, may have some questions.

[dmdymov]

Guys, I cant help you with publishing, but 4.13 is very much needed somewhere. I've used com.innoventsolutions 4.8 for years but cant find 4.13 anywhere.

🙏🙏🙏

[dmdymov]

@SteveSchafer-Innovent maybe you could publish 4.13 to the group com.innoventsolutions until this issue is resolved?

[ruspl-afed]

Do you mean some manual publishing process @dmdymov until we have automation for it? BTW, what is your use case?

[dmdymov]

Can I get any clarification on why my comment was deleted, please?

[wimjongman]

Sorry Dmytry, it was a glitch in the GitHub UI where I saw your comment twice. Please repost.

[dmdymov]

@ruspl-afed Yes, I am talking about the manual publishing process as a temporary workaround. We have an offline private maven repo and I want it to be a mirror as much as possible.

[wimjongman]

I see we have published 4.9 last year. I will take a look at 4.13

[GregoireW]

Hello, I see here (https://ci.eclipse.org/birt/job/Publish%20to%20Maven%20Central/33/ ) a job with version 4.13 but this is not showing in maven central. Is the process been cancelled ? Thank you

[wimjongman]

It has not been cancelled but I did not find the time. We are releasing 4.14 soon so we can try again with that one.

Ed @merks does your build process also have capabilities to publish to a maven repo?

[merks]

No.

I had a brief glance at the jobs that seem to be maven related, but I don't understand what they are doing or even where they are publishing. This jobs seems to fail but I don't understand why:

https://ci.eclipse.org/birt/job/maven_test/

And I don't see anything to suggest there haven been updates here since 4.9.0:

https://repo1.maven.org/maven2/org/eclipse/birt/org.eclipse.birt.core/

So I don't really know where things are being published.

[wimjongman]

AFAIK, we never had an official automatic publish. I believe that 4.9 was published by Innovent manually. The maven_test build job was probably created by me one on of my attempts to get it going. I failed because lack of time.

[merks]

I know that for other projects the publishing process is relatively complicated and it isn't just a "dump" of the pom structure of the project, which appears to be what I see for 4.9. at maven central. E.g., the Platform's process is quite complex:

https://ci.eclipse.org/releng/view/Publish%20to%20Maven/

I'm not sure anyone should expect this to happen "for free" and that someone else will do it for free because it's a very significant investment of time to do this properly. E.g., all the OSGi dependencies really need to be mapped properly to dependencies in the pom.xml, otherwise the result just won't be usable. I think there are weeks of effort involved in doing such a thing properly and 25 thumbs up don't put a crumb bread on the table...

[christophersavory]

looks like the command to release isn't correct

/opt/tools/apache-maven/latest/bin/mvn -f pom.xml -Dversion=4.13.0 -Dmaven.test.skip=true -Drelease.base.dir=/home/data/httpd/download.eclipse.org/birt clean package --no-transfer-progress -U -Pmaster,build-server,release

I don't see a release profile in the pom.