search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.82k stars 13.92k forks source link

buildMaven duplicate dependencies don't build any longer #200312

Open zeratax opened 1 year ago

zeratax commented 1 year ago

Describe the bug

A maven project doesn't build any longer probably since #100634

https://github.com/zeratax/nur-packages/blob/39e82f546c9dca6431d15d9ca179ef912104a8cd/pkgs/bukkit-plugins/harbor/default.nix

which is a derivation for https://github.com/nkomarn/harbor/tree/1.6.3

not sure if this is so much a buildMaven issue or a issue with the project.json which it uses so i filed an issue here too: https://github.com/NixOS/mvn2nix-maven-plugin/issues/30

Steps To Reproduce

Steps to reproduce the behavior:

  1. create a project.json with https://github.com/NixOS/mvn2nix-maven-plugin
  2. build the project

I assume this requires a maven project with cyclic dependencies

Additional context

these 2 derivations will be built:
  /nix/store/m2pdqkz55q90c56nibp2dxg80jbv49rc-maven-repository.drv
  /nix/store/76fylvki3d6y79n3y3glfvgm9dm48szm-harbor-1.6.3.drv
building '/nix/store/m2pdqkz55q90c56nibp2dxg80jbv49rc-maven-repository.drv'...
ln: failed to create symbolic link 'org/spigotmc/spigot-api/1.14.4-R0.1-SNAPSHOT/maven-metadata-spigotmc-repo.xml': File exists
error: builder for '/nix/store/m2pdqkz55q90c56nibp2dxg80jbv49rc-maven-repository.drv' failed with exit code 1;
       last 1 log lines:
       > ln: failed to create symbolic link 'org/spigotmc/spigot-api/1.14.4-R0.1-SNAPSHOT/maven-metadata-spigotmc-repo.xml': File exists
       For full logs, run 'nix log /nix/store/m2pdqkz55q90c56nibp2dxg80jbv49rc-maven-repository.drv'.
error: 1 dependencies of derivation '/nix/store/76fylvki3d6y79n3y3glfvgm9dm48szm-harbor-1.6.3.drv' failed to build

Notify maintainers

@fzakaria

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[nixos@nixos:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.68.1-microsoft-standard-WSL2, NixOS, 22.11 (Raccoon), 22.11pre424584.667e5581d16`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.0`
 - channels(root): `"home-manager-22.05.tar.gz, nixos, nixos-unstable"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`
fzakaria commented 1 year ago

here is the relevant code

  repo = linkFarm "maven-repository" (lib.forEach dependencies (dependency: {
    name = dependency.layout;
    path = dependency.drv;
  }));

That must mean the dependencies are present more than once?

I'll try to investigate this weekend unless you had made progress.

zeratax commented 1 year ago

The spigot-api dependecy seems to be twice in the project.json: spigot-api

but it also complains about this metadata file if i remove one of the duplicate spigot-api dependencies:

error: builder for '/nix/store/rxjwq5prv7sjam1id8wmq24mc92m9qsi-maven-repository.drv' failed with exit code 1;
       last 1 log lines:
       > ln: failed to create symbolic link 'org/bukkit/bukkit/1.12.2-R0.1-SNAPSHOT/maven-metadata-spigotmc-repo.xml': File exists

That metadata is defined by both a pom and jar with the same name:

bukkit

removing additionally either one of the bukkit dependencies and one of the duplicate spigot-api dependencies it will then complain about spigot-api missing, even though it to me looks like it is just a duplicate?

[ERROR] Failed to execute goal on project Harbor: Could not resolve dependencies for project 
xyz.nkomarn:Harbor:jar:1.6.3: The following artifacts could not be resolved: org.spigotmc:spigot-api:jar:1.14.4-R0.1-SNAPSHOT, net.md-5:bungeecord-chat:jar:1.13-SNAPSHOT, io.papermc:paperlib:jar:1.0.2, org.bukkit:bukkit:jar:1.12.2-R0.1-SNAPSHOT: 
    Cannot access spigotmc-repo (https://hub.spigotmc.org/nexus/content/repositories/snapshots/) in offline mode and the artifact org.spigotmc:spigot-api:jar:1.14.4-R0.1-20191210.224154-92 has not been downloaded from it before.
    -> [Help 1]

Edit: those tests are with the project file for 1.6.3 of harbor on main is 1.6.4 right now. But the issue are essentially the same, just with more duplicates.

Easy to see find the duplicate metadata files via: 

$ curl -sL "https://github.com/zeratax/nur-packages/blob/master/pkgs/bukkit-plugins/harbor/project-info.json?raw=true" | 
    jq ".dependencies |
      group_by(.metadata)  | 
      map(select(length>1) | .[0])"
Output: ```json [ { "artifactId": "maven-resources-plugin", "groupId": "org.apache.maven.plugins", "version": "2.6", "classifier": "", "extension": "jar", "dependencies": [ { "artifactId": "maven-plugin-api", "groupId": "org.apache.maven", "version": "2.0.6", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "maven-project", "groupId": "org.apache.maven", "version": "2.0.6", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "maven-core", "groupId": "org.apache.maven", "version": "2.0.6", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "maven-artifact", "groupId": "org.apache.maven", "version": "2.0.6", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "maven-settings", "groupId": "org.apache.maven", "version": "2.0.6", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "maven-model", "groupId": "org.apache.maven", "version": "2.0.6", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "maven-monitor", "groupId": "org.apache.maven", "version": "2.0.6", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "plexus-container-default", "groupId": "org.codehaus.plexus", "version": "1.0-alpha-9-stable-1", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "maven-plugin-annotations", "groupId": "org.apache.maven.plugin-tools", "version": "3.1", "classifier": "", "extension": "jar", "scope": "provided", "optional": false, "exclusions": [] }, { "artifactId": "plexus-utils", "groupId": "org.codehaus.plexus", "version": "2.0.5", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "maven-filtering", "groupId": "org.apache.maven.shared", "version": "1.1", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "plexus-interpolation", "groupId": "org.codehaus.plexus", "version": "1.13", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "maven-plugin-testing-harness", "groupId": "org.apache.maven.shared", "version": "1.0-beta-1", "classifier": "", "extension": "jar", "scope": "test", "optional": false, "exclusions": [] }, { "artifactId": "commons-io", "groupId": "commons-io", "version": "1.4", "classifier": "", "extension": "jar", "scope": "test", "optional": false, "exclusions": [] }, { "artifactId": "maven-plugins", "groupId": "org.apache.maven.plugins", "version": "23", "classifier": "", "extension": "pom", "scope": "compile", "optional": false, "exclusions": [] } ], "authenticated": false, "url": "https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.jar", "sha1": "dd093ff6a4b680eae7ae83b5ab04310249fc6590", "relocations": [] }, { "artifactId": "bungeecord-chat", "groupId": "net.md-5", "version": "1.13-20200123.020938-2", "classifier": "", "extension": "jar", "dependencies": [ { "artifactId": "gson", "groupId": "com.google.code.gson", "version": "2.8.0", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "junit", "groupId": "junit", "version": "4.12", "classifier": "", "extension": "jar", "scope": "test", "optional": false, "exclusions": [] }, { "artifactId": "guava", "groupId": "com.google.guava", "version": "20.0", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "findbugs-annotations", "groupId": "com.google.code.findbugs", "version": "3.0.1", "classifier": "", "extension": "jar", "scope": "provided", "optional": false, "exclusions": [] }, { "artifactId": "lombok", "groupId": "org.projectlombok", "version": "1.18.4", "classifier": "", "extension": "jar", "scope": "provided", "optional": false, "exclusions": [] }, { "artifactId": "bungeecord-parent", "groupId": "net.md-5", "version": "1.13-SNAPSHOT", "classifier": "", "extension": "pom", "scope": "compile", "optional": false, "exclusions": [] } ], "unresolved-version": "1.13-SNAPSHOT", "repository-id": "papermc-repo", "metadata": { "url": "https://papermc.io/repo/repository/maven-public/net/md-5/bungeecord-chat/1.13-SNAPSHOT/maven-metadata.xml", "sha1": "0c0cf36832ce37f41a8abfc063e9b4b832add1bc" }, "authenticated": false, "url": "https://papermc.io/repo/repository/maven-public/net/md-5/bungeecord-chat/1.13-SNAPSHOT/bungeecord-chat-1.13-20200123.020938-2.jar", "sha1": "0d0cf49e1752e31eb09eb5d57d5f0835b81e5fe6", "relocations": [] }, { "artifactId": "spigot-api", "groupId": "org.spigotmc", "version": "1.14.4-R0.1-20191210.224154-92", "classifier": "", "extension": "jar", "dependencies": [ { "artifactId": "commons-lang", "groupId": "commons-lang", "version": "2.6", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "guava", "groupId": "com.google.guava", "version": "21.0", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "gson", "groupId": "com.google.code.gson", "version": "2.8.0", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "bungeecord-chat", "groupId": "net.md-5", "version": "1.13-SNAPSHOT", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "snakeyaml", "groupId": "org.yaml", "version": "1.23", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "annotations-java5", "groupId": "org.jetbrains", "version": "17.0.0", "classifier": "", "extension": "jar", "scope": "provided", "optional": false, "exclusions": [] }, { "artifactId": "junit", "groupId": "junit", "version": "4.12", "classifier": "", "extension": "jar", "scope": "test", "optional": false, "exclusions": [] }, { "artifactId": "hamcrest-library", "groupId": "org.hamcrest", "version": "1.3", "classifier": "", "extension": "jar", "scope": "test", "optional": false, "exclusions": [] }, { "artifactId": "asm-tree", "groupId": "org.ow2.asm", "version": "7.1", "classifier": "", "extension": "jar", "scope": "test", "optional": false, "exclusions": [] } ], "unresolved-version": "1.14.4-R0.1-SNAPSHOT", "repository-id": "spigotmc-repo", "metadata": { "url": "https://papermc.io/repo/repository/maven-public//org/spigotmc/spigot-api/1.14.4-R0.1-SNAPSHOT/maven-metadata.xml", "sha1": "529696d3790000c99bbaa1fc7c33be20bd3675c8" }, "authenticated": false, "url": "https://hub.spigotmc.org/nexus/content/repositories/snapshots//org/spigotmc/spigot-api/1.14.4-R0.1-SNAPSHOT/spigot-api-1.14.4-R0.1-20191210.224154-92.jar", "sha1": "1cce9eb1461932da654aa193bccc760cf848482a", "relocations": [] }, { "artifactId": "bukkit", "groupId": "org.bukkit", "version": "1.12.2-R0.1-20180712.012114-155", "classifier": "", "extension": "jar", "dependencies": [ { "artifactId": "commons-lang", "groupId": "commons-lang", "version": "2.6", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "json-simple", "groupId": "com.googlecode.json-simple", "version": "1.1.1", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "guava", "groupId": "com.google.guava", "version": "21.0", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "gson", "groupId": "com.google.code.gson", "version": "2.8.0", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "snakeyaml", "groupId": "org.yaml", "version": "1.19", "classifier": "", "extension": "jar", "scope": "compile", "optional": false, "exclusions": [] }, { "artifactId": "junit", "groupId": "junit", "version": "4.12", "classifier": "", "extension": "jar", "scope": "test", "optional": false, "exclusions": [] }, { "artifactId": "hamcrest-library", "groupId": "org.hamcrest", "version": "1.3", "classifier": "", "extension": "jar", "scope": "test", "optional": false, "exclusions": [] } ], "unresolved-version": "1.12.2-R0.1-SNAPSHOT", "repository-id": "spigotmc-repo", "metadata": { "url": "https://papermc.io/repo/repository/maven-public//org/bukkit/bukkit/1.12.2-R0.1-SNAPSHOT/maven-metadata.xml", "sha1": "eac75895feaa545f4b60320be29c3afcf79b40cc" }, "authenticated": false, "url": "https://hub.spigotmc.org/nexus/content/repositories/snapshots//org/bukkit/bukkit/1.12.2-R0.1-SNAPSHOT/bukkit-1.12.2-R0.1-20180712.012114-155.jar", "sha1": "ebb39bc230e588e660c100b507f55af34ba7ea70", "relocations": [] } ] ```