search

repaint-io / maven-tiles

Injecting maven configurations by composition rather than inheritance
154 stars 32 forks source link

'dependencies.dependency.version' for xxx is missing #104

Open tomaskir opened 5 years ago

tomaskir commented 5 years ago

Hi there.

I am testing out tiles because I need to bring in Spring Boot (which is normally done through a parent pom) into a project that already has a different parent pom.

I can't touch Spring poms to make our parent into their parent.
I can't make the Spring Boot parent into a parent of our parent (that would bring Spring Boot into all our projects).
So I am hoping to achieve this with tiles.

I have successfully wrote and mvn installed all the tiles I need, and I can inject them into the project's pom.

However, when trying to mvn compile, or even mvn effective-pom, it seems the tiles are not properly merged into the pom.

[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project xxx:2.0.0-SNAPSHOT (/home/xxx/pom.xml) has 8 errors
[ERROR]     'dependencies.dependency.version' for javax.servlet:javax.servlet-api:jar is missing. @ line 131, column 21
[ERROR]     'dependencies.dependency.version' for org.springframework.security:spring-security-core:jar is missing. @ line 138, column 21
[ERROR]     'dependencies.dependency.version' for org.springframework.boot:spring-boot-starter-data-jpa:jar is missing. @ line 144, column 21
...

All of these are declared as <dependency> in the project's pom, without specifying a version. The version is supposed to come from Spring Boot's <dependencyManagement> section, which is normally brought in from their parent.

I have moved the <dependencyManagement> into a tile, and this tile is properly brought in in the <tiles> section. (I know this is listed as a smell in the tiles documentation, but this is how Spring Boot does things - I don't have much of a choice)

Few questions:

There doesn't seem to be much documentation how tiles actually integrates into Maven (when in the lifecycle tiles does it's thing, how to inspect the output pom that tiles generates before it goes to Maven etc.)

Any inside is appreciated, thanks!

tomaskir commented 5 years ago

@talios any chance for a comment?

talios commented 5 years ago

Tomas,

Sorry I've not responded sooner - regarding the merging - do you see the tile being logged as being injected at all?

From a glance at the code we should be copying over the dependencyManagement section so in theory this should work. Basically each tile in injected as parents between your project and the parent/superpom (so order can matter).

So that I can build up a test, am i correct that:

?

Sent from my iPhone

On 29/07/2019, at 11:46 PM, Tomas Kirnak notifications@github.com wrote:

@talios any chance for a comment? :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

tomaskir commented 5 years ago

I have made an example where you can reproduce the issue: https://github.com/tomaskir/maven-tile-depmng

You should just be able to clone this and run ./replicate.sh to see the issue. You can import the pom.xml in the root of the project to your favourite IDE to see how it's strucuted :)

tomaskir commented 5 years ago

Maybe a bit more on what we are trying to achieve:

Our real setup is of course a bit more complicated, but I tried to make the example as simple as I could while still replicating the issue.

tomaskir commented 5 years ago

@talios I apologize for pushing, but any chance for a comment?

talios commented 5 years ago

On 5 Aug 2019, at 10:48, Tomas Kirnak wrote:

@talios I apologize for pushing, but any chance for a comment?

Hi Tomas,

Sorry for not replying sooner - I pulled down your sample repo the other night and did a bit of digging - several immediate things did pop up, and unfortunately - a flash back to a limitation I encountered writing another lifecycle based plugin.

The main one - the resolution of ${some.version} from a dependency. Unfortunately, from what I gather from stepping thru, the initial maven model is built, and validated, before the tiles plugin is executed.

Unfortunately - when we have a <dependency> without a version element, and no <dependencyManagement> (or a declared parent with one) then that fails that validation. Similarly, when we have a <version>${some.version}</version> declared, but no declaration of the property in either the pom, or parent - again - validation failure.

From what I can gather, the validation looks specifically at the parent declared in the pom early on in maven's operation - unfortunately this is where it hurts us.

One potential resolution to this is to not use a dependency management section, but just dependencies that magically come into your project when you add the tile (not really ideal).

Another potential resolution, which also isn't entirely ideal, which I was trying to find some further time to explore (based on code I originally used in another plugin) was to extend the limited support we have for declaring the tiles extension via the .mvn/extensions.xml file, see:

From here we can hook in to provide properties values to that top level/initial model resolution from elsewhere (i.e. the tile definition). Unfortunately, extensions in this file AFAIK don't have any config, so we'd need to lookup the associated pom.xml and parse out the tiles.

Note these extensions are run ONCE per maven instance (which break within IntelliJ etc. as they maintain a long running maven process) so aren't really ideal for use in reactor projects.

This leads to another issue I noticed in your sample project - the client project is getting a tile-plugin definition from a parent pom - which, I'm not sure, but that could cause a lot of confusion in the processing (maven would have already resolved a parent in order to pick up the tiles plugin, which we'd then start reparenting and... I'm not entirely sure what would happen here - but it could be a debugging nitemare).

Ideally each project in the reactor should declare it's own tiles-plugin.

I start think about the .mvn/extensions.xml solution and how, that might be able to play in the work addressing Rob's tile-fragment's and my idea of a special fragment lifecycle/packaging, I was thinking something along the lines of special casing property definitions for a tile which could be loaded by the extension to inject at the root - a tile-skirting or such construct ( stored as a simple tile.properties artefact along side the tile.xml).

I'm still thinking...

"The ease with which a change can be implemented has no relevance at all to whether it is the right change for the (Java) Platform for all time." — Mark Reinhold.

Mark Derricutt http://www.theoryinpractice.net http://www.chaliceofblood.net http://plus.google.com/+MarkDerricutt http://twitter.com/talios http://facebook.com/mderricutt

tomaskir commented 5 years ago

Unfortunately - when we have a <dependency> without a version element, and no <dependencyManagement> (or a declared parent with one) then that fails that validation. Similarly, when we have a <version>${some.version}</version> declared, but no declaration of the property in either the pom, or parent - again - validation failure.

From what I can gather, the validation looks specifically at the parent declared in the pom early on in maven's operation - unfortunately this is where it hurts us.

So this basically means that transforming Spring Boot parent .poms to tiles would not be possible.

One potential resolution to this is to not use a dependency management section, but just dependencies that magically come into your project when you add the tile (not really ideal).

Indeed this is not ideal - we were specifically hoping to NOT modify the Spring parents, other than copy-paste them into tiles. This is so we can easily verify if we have correct contents of the tile (as compared to the original Spring Boot pom).

This would allow developers to correctly work with Spring Boot without even knowing we are using Tiles instead of the Spring Boot parent.

Another potential resolution ... via the .mvn/extensions.xml file ... Note these extensions are run ONCE per maven instance (which break within IntelliJ etc. as they maintain a long running maven process) so aren't really ideal for use in reactor projects.

This also doesn't sound like a good solution.

This leads to another issue I noticed in your sample project - the client project is getting a tile-plugin definition from a parent pom

We were hoping to declare this in the parent to avoid repetitive code in all .poms, and to enforce Tiles version from the parent. (instead of having to change all poms to update Tiles version)

All in all, it sound like we are hitting limitations of Maven for what we want to do. (or we are forced to the limitations with how Spring Boot integrates with Maven)

Do you have any idea if we could do something like the sample project I provided in Gradle?

drekbour commented 3 years ago

This is the single worst issue I've found in using tiles (appreciate it's not bug but an upstream limitation). It completely fails user expectations that <properties> merge and <version>${some.version}</version> works in their pom.

Is there any active plan on solving it (or another issue raised somewhere)?

rvowles commented 3 years ago

Hi there,

Could you perhaps be a bit more descriptive? I use properties in my tiles, with them being expanded, so I think an example would be helpful?

If it's affecting you it will probably affect us at some point 🙂

On Wed, May 19, 2021, 11:15 PM drekbour @.***> wrote:

This is the single worst issue I've found in using tiles (appreciate it's not bug but an upstream limitation). It completely fails user expectations that merge and ${some.version} works in their pom.

Is there any active plan on solving it (or another issue raised somewhere)?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/repaint-io/maven-tiles/issues/104#issuecomment-844002533, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAANTWAO6FWUC4MR67QNRZLTOOMWZANCNFSM4IGS7CXA .

talios commented 3 years ago

As @rvowles says - an example would be useful. I know Spring Boot makes use of BOM style dependency management - and usually that's pulled into a project as a dependency using the import scope.

Properties for versions work, but only if you actually mention the <version>${some.version}</version> in your pom (and if that's in a tile, that tile needs to declare the property as well).

drekbour commented 3 years ago

@talios FYI, extensions.xml cannot have config but other plugins simply look for a config file in .mvn once they are bootstrapped. https://github.com/jgitver/jgitver-maven-plugin/blob/fbcc04cc96cb9f06710e14d95522465afff4770c/src/main/java/fr/brouillard/oss/jgitver/cfg/ConfigurationLoader.java#L48

@rvowles It's summarised above by

The initial maven model is built, and validated, before the tiles plugin is executed .... when we have a <dependency> without a version element, and no <dependencyManagement> (or a declared parent with one) then that fails that validation. Similarly, when we have a <version>${some.version}</version> declared, but no declaration of the property in either the pom, or parent - again - validation failure.

Maven BOM users (a prevalent pattern including any modern usages of Spring Boot which really emphasises it) expect that their app should not ever mention a version for something managed via the BOM (unless you have a burning requirement to pin something):

<dependencies>
  <dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
  </dependency>
</dependencies>

These two patterns are incompatible and make it harder to fully embrace tiles for a lot of projects.

Would be nice if this issue were renamed or, better, a new feature created to encapsulate 'Tiles lifecycle plugin' and what it would add to the capabilities.

rvowles commented 3 years ago

Spring Boot is not something that anyone should look for any reason at, it is a fractal of bad design choices, from bottom to top! The abuse of dependencyManagement management is always one that has really, really bothered me.

Maven Tiles was always designed as the anti-thesis of this pattern, so its right that it is hard to smuge the two things together, it is focused soley on solving plugin composition, dependency management is easily managed by other patterns (such as composites). Mark and I have always been about trying to find the best way to do things properly, even if Mark tends to take it a bit far :-) Having been working with a Gradle build, which is an atrocious mess, you will find us less enthusiastic about following Springs path to madness sorry 😂

That said, the concept of being able to auto-update plugin versions I think is useful, and its actually something we could potentially bake into the plugin itself or borg from the Versions plugin. Plugins have their own versions and they have dependency libraries you can add to them, so creating a solution where plugins and their respective dependencies can be kept up to date is a desirable feature.

drekbour commented 3 years ago

fractal of bad design

Superb :) Not heard that one in a while! I would say to ignore the Spring angle except for two lessons: BOM's are ubiquitous and Spring Boot is a jack-of-all-trades. A more UNIX do-one-thing-well philosophy applied to tiles would be "tile stuff and then get out of the way". This derives the goal of not restricting users in what vanilla Maven features can be used alongside or inside a tile and of ability to update tiles using any mojo they choose.

It feels like a .mvn lifecycle extension allowing "early tiling" would make these viable but I see it's a lot of effort for a usage model you don't wholly support! However, IMHO it is "the next step" should tiles want to advance in capability.

Rohan427 commented 2 years ago

I have similar issues with Spring Boot and I'm now re-thinking my use of it.

I have a Java payment processing library (a SOAP API that allows an external web site to provide online credit card payments) that works just fine. It uses an older version of Hibernate (which I'm not a fan of) and Spring Framework 5.x (I like Springs dependency injection). The only reason I'm looking at Spring Boot is because employers (including contracts) want people that have experience with it, so I am working on getting that experience (and NOT liking it). In my 30+ years of experience, I have found Maven especially and Spring Boot to be more trouble than they are worth. Make and Ant always worked just fine and both are very configurable and I've never had to fight with them like I do with Maven (or Gradle).

Anyway, my issue is similar - no dependency version causes NetBeans 12.6 (and NetBeans 8.2) to fail to import a Spring Boot project without a version. Since Spring does not support anything other than Maven and Gradle (not well anyway), "upgrading" my payment API (which uses Ant) to use Spring Boot has taken far more time then it has to upgrade from EclipsLink to HIbernate, then upgrade the version of Hibernate it uses, or even to change from Hibernate to Oracle's ADF ORM.

lfvjimisola commented 1 year ago

Another organization that stumbled onto this one as well.

Out of curiousity, I can see that there are new versions released but it seems a they are normally dependency upgrades and bug fixes rather than additional fixes and features. What is the roadmap for Maven Tiles?

Our organization had really high hopes of Maven Tiles but it might not suit all circumstances. If there is an known issue/limitations with BOMs then perhaps it could be mentioned in the README?

drekbour commented 1 year ago

If there is an known issue/limitations with BOMs then perhaps it could be mentioned in the README? Amen.

Warning: I have not personally confirmed the below points about it being ok to use nonsense placeholders. If someone does determin this either way then I can update this comment.

Tiles doesn't (cleanly) support

Placeholders are required to not be missing during earliest POM initialisation which is more about syntactical correctness, they will be replaced with values from your tiles before becoming active. These placeholders must be set in a parent project (since tiles will, by design, never override anything in the local project).

Warning: These workarounds provide the ability to significantly bypass the reproducible-builds property expected by most users/organisations. Tiles imported with open-ended version ranges ([1.0,2.0)) mean the same commit could produce totally different dependencies when executed after a tile is updated.

rvowles commented 1 year ago

Open ended dependencies of any kind provide that, but tiles are not intended to be used with dependencies, only for plugins. Composites are used for dependencies, my open source project FeatureHub uses both halves, if that's of use. But we also use trunk based development, differential builds and a monorepo.

talios commented 1 year ago

I did start looking into this earlier today, but ended up going to pub.

However - as @rvowles says tiles is primarily about behaviour - via plugins.

Looking at the maven-tile-depmng mentioned above, your right - that doesn't work. Given the <dependencyManagement> declared is in a parent, it should work, if that parently chain was there prior to the validation step - which I believe occurs earlier than the extensions kick in.

So stubbing the out may work. An alternative, which I'm going to look at tomorrow, is to move the <dependencyManagement> block from the tile.xml to the pom.xml - then add a import <dependency/> block into the child.

My hope then, is that this provides a "best of both worlds" approach - tiling in the behavior, and importing the dependency.

Unfortunately - this doesn't quie work (so I just found out):

[WARNING] 'dependencyManagement.dependencies.dependency.type' for com.github.tomaskir:spring-boot-1.5-tile:xml must be 'pom' to import the managed dependencies. @ line 24, column 23

That project is.... quite a mess tho, esp when trying to tease things apart and see what is actually failing.

The top level pom declares the parent, but also includes the parent (which I believe does get resolved, just confusing), and the moment you don't include that parent anywhere - nothing works due to the lack of groupId's being repeated (again, thats allowed due to the parent). However, the intended use of tiles is to not use parents (if possible).

I noticed when trying to build, nothing seemed to be triggering the build of the parent.

Two options come to mind:

I think the first option maybe the best option, maintaining the versions is still left up to the tile author by whatever means they see fit (aka not part of the tiles-maven-plugin itself).

lfvjimisola commented 1 year ago

@talios Thanks for comment. Our team just decided that we won't use Maven Tiles as of now. We are afraid that we end up with other stuff that won't work for us (such as integration with VS Code).