OpenWIS / openwis

http://openwis.github.io/openwis
GNU General Public License v3.0
11 stars 15 forks source link

OpenWIS Maven Cache Build Issue #398

Open mgiannoni opened 2 years ago

mgiannoni commented 2 years ago

OpenWIS Maven Cache Build Issue

Related Issue #390 : Problem openwis 3.14.16

Building OpenWIS "master" branch with a clean Maven cache may encounter problems retrieving dependencies located at ForgeRock. The OpenAM integration and the associated ForgeRock dependencies are continually being "closed-source" as the company takes their repositories off-line.

The Maven "dependencies" profile and associated "openwis-libs" module resolve this issue by storing legacy dependencies directly in the repository. However this creates a tread-mill where future artifacts that go off-line must be integrated into the "openwis-libs" module.

[INFO] openwis-securityservice ............................ SUCCESS [ 0.010 s] [INFO] openwis-securityservice-war ........................ SUCCESS [ 56.948 s] [INFO] openwis-securityservice-utils-populate-ldap ........ SUCCESS [ 51.380 s] [INFO] openwis-stagingpost ................................ SUCCESS [ 0.019 s] [INFO] OpenWIS module ..................................... FAILURE [01:39 min] [INFO] openwis-dependencies ............................... SKIPPED [INFO] openwis-tools ...................................... SKIPPED [INFO] load-test-data ..................................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 23:23 min [INFO] Finished at: 2022-06-15T10:23:12+02:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project openwis-portal: Could not resolve dependencies for project org.geonetwork-opensource:openwis-portal:war:3.14.16-SNAPSHOT: Failed to collect dependencies at org.forgerock.openam:openam-federation-library:jar:13.0.0 -> xerces-J:xml-resolver:jar:2.11.0: Failed to read artifact descriptor for xerces-J:xml-resolver:jar:2.11.0: Could not transfer artifact xerces-J:xml-resolver:pom:2.11.0 from/to forgerock-staging-repository (http://maven.forgerock.org/repo/releases): Authentication failed for http://maven.forgerock.org/repo/releases/xerces-J/xml-resolver/2.11.0/xml-resolver-2.11.0.pom 401 Unauthorized -> [Help 1] [ERROR]

Updates that restore the build look like: Note: This specific update resolved several issues unrelated to the error message.

$ git status On branch master Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory)

  modified:   openwis-libs/pom.xml

Untracked files: (use "git add ..." to include in what will be committed)

  openwis-libs/libs/jaxrpc-impl-1.1.3_01-041406.jar
  openwis-libs/libs/jaxrpc-impl-1.1.3_01-041406.pom

no changes added to commit (use "git add" and/or "git commit -a")

mgiannoni commented 1 year ago

Changes Staged To Feature Branch "Issue_398_MVN_Cache_Build_Error"

$ git fetch origin
$ git reset --hard origin/master
HEAD is now at 10f470e Merge pull request #380 from OpenWIS/solve-travis-issues-after-migrate-to-dot-com
$ git checkout -b Issue_398_MVN_Cache_Build_Error
Switched to a new branch 'Issue_398_MVN_Cache_Build_Error'

$ git status
# On branch Issue_398_MVN_Cache_Build_Error
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#   modified:   openwis-libs/pom.xml
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   openwis-libs/libs/jaxrpc-impl-1.1.3_01-041406.jar
#   openwis-libs/libs/jaxrpc-impl-1.1.3_01-041406.pom

$ git add openwis-libs/libs/jaxrpc-impl-1.1.3_01-041406.jar
$ git add openwis-libs/libs/jaxrpc-impl-1.1.3_01-041406.pom
$ git commit -a
[Issue_398_MVN_Cache_Build_Error d0d62f0] Issue #398 OpenWIS Maven Cache Build Issue
 3 files changed, 70 insertions(+), 3 deletions(-)
 create mode 100644 openwis-libs/libs/jaxrpc-impl-1.1.3_01-041406.jar
 create mode 100644 openwis-libs/libs/jaxrpc-impl-1.1.3_01-041406.pom

$ git push -u origin Issue_398_MVN_Cache_Build_Error
Username for 'https://github.com': mgiannoni
Password for 'https://mgiannoni@github.com': 
Counting objects: 11, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 1.92 MiB | 0 bytes/s, done.
Total 7 (delta 3), reused 0 (delta 0)
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
remote: 
remote: Create a pull request for 'Issue_398_MVN_Cache_Build_Error' on GitHub by visiting:
remote:      https://github.com/OpenWIS/openwis/pull/new/Issue_398_MVN_Cache_Build_Error
remote: 
To https://github.com/OpenWIS/openwis.git
 * [new branch]      Issue_398_MVN_Cache_Build_Error -> Issue_398_MVN_Cache_Build_Error
Branch Issue_398_MVN_Cache_Build_Error set up to track remote branch Issue_398_MVN_Cache_Build_Error from origin.
zhangz555 commented 1 year ago

For issues in 3.14, the group at MDL/NOAA will take the approach Marc presented, if we do not need to maintain 3.14 for a longer time. We can quickly wrap up the fixing process by extracting and using some artifacts from the back-up file Marc stores.

zhangz555 commented 1 year ago

Similar issues also occur in version 3.15. For 3.15, I would suggest that we recommend to AKKA setting up a maven repository which can be internally open to OpenWIS users. From the last meeting, we now know that Akka has already dropped the "dependencies" profile in their design. This maven repository can be built up by utilizing the backup file Marc stores as a source.

zhangz555 commented 1 year ago

After Marc pushes his changes into the specific feature branch, I would suggest that YvesG from the French group take a test build on their platform. If everything goes well, we would merge the changes into master.

zhangz555 commented 1 year ago

YvesG performed the test build on their platform for this specific feature branch. The build is ok.

zhangz555 commented 1 year ago

I have requested merge/pull request. Once travis build chain finishes successful, review and approval will be needed to make the fix official.

mgiannoni commented 1 year ago

Branch 3.16 has a problem when I try to build using the library module to include the ForgeRock libraries:

mvn clean install -P dependencies,openwis -DskipTests -Dfile.encoding=UTF-8

However the build is successful if I remove the dependencies module and copy in the Forgerock libraries from an older Maven Cache I have saved from earlier builds.

I get the following error:

$ mvn clean install -P dependencies,openwis -DskipTests -Dfile.encoding=UTF-8 [INFO] Scanning for projects... [ERROR] [ERROR] Some problems were encountered while processing the POMs: [FATAL] Non-resolvable parent POM for io.openwis.libs:openwis-libs:[unknown-version]: Could not find artifact io.openwis:openwis:pom:3.14.16-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 4, column 11 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ io.openwis.management.service:openwis-management-service-common:[unknown-version],/home/openwis/OpenWIS/openwis/openwis-management/openwis-management-service/openwis-management-service-common/pom.xml, line 48, column 12 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-war-plugin is missing. @ io.openwis.securityservice:openwis-securityservice-war:[unknown-version], /home/openwis/OpenWIS/openwis/openwis-securityservice/openwis-securityservice-war/pom.xml, line 89, column 21 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-war-plugin is missing. @ io.openwis.stagingpost:openwis-stagingpost:[unknown-version], /home/openwis/OpenWIS/openwis/openwis-stagingpost/pom.xml, line 32, column 21 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-war-plugin is missing. @ line 174, column 21 [WARNING] 'build.plugins.plugin.version' for org.codehaus.mojo:exec-maven-plugin is missing. @ io.openwis.tools:load-test-data:[unknown-version], /home/openwis/OpenWIS/openwis/openwis-tools/load-test-data/pom.xml, line 109, column 23 @ [ERROR] The build could not read 1 project -> [Help 1] [ERROR] [ERROR] The project io.openwis.libs:openwis-libs:[unknown-version] (/home/openwis/OpenWIS/openwis/openwis-libs/pom.xml) has 1 error [ERROR] Non-resolvable parent POM for io.openwis.libs:openwis-libs:[unknown-version]: Could not find artifact io.openwis:openwis:pom:3.14.16-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 4, column 11 -> [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

I believe I have the most recent repo for this build:

commit 15f363e857e482694289ad0ecb37de6d32848812 (HEAD -> openwis-3.16, origin/openwis-3.16) Author: brahim [brahim.handala@akka.eu](mailto:brahim.handala@akka.eu) Date: Mon Dec 12 14:16:04 2022 +0100

BHA: Admin portal  create user  bug fix

commit 5667b4b2e765b21ad232da548aa451a2df4997ad Author: brahim [brahim.handala@akka.eu](mailto:brahim.handala@akka.eu) Date: Fri Dec 9 15:26:48 2022 +0100

BHA:  GenerateSPConfFiles.zip fix

commit 407d66c45f62d68d119383893de0debb89c145a6 Author: brahim [brahim.handala@akka.eu](mailto:brahim.handala@akka.eu) Date: Wed Dec 7 15:04:31 2022 +0100

BHA: Add openwis-3.16 installion  guide

commit bcee0542e98c12d18afe5cd554a01215f1db9928 Author: brahim [brahim.handala@akka.eu](mailto:brahim.handala@akka.eu) Date: Wed Dec 7 15:01:35 2022 +0100

BHA: add solr  configuration file
zhangz555 commented 1 year ago

Hi Marc, I saw this error when seeing Brahim testing ci/cd. I think this issue should have been solved since his ci/cd appears successful on travis. --Zhan

On Wed, Jan 4, 2023 at 12:42 PM Marc Giannoni @.***> wrote:

Branch 3.16 has a problem when I try to build using the library module to include the ForgeRock libraries:

mvn clean install -P dependencies,openwis -DskipTests -Dfile.encoding=UTF-8

However the build is successful if I remove the dependencies module and copy in the Forgerock libraries from an older Maven Cache I have saved from earlier builds.

I get the following error:

$ mvn clean install -P dependencies,openwis -DskipTests -Dfile.encoding=UTF-8 [INFO] Scanning for projects... [ERROR] [ERROR] Some problems were encountered while processing the POMs: [FATAL] Non-resolvable parent POM for io.openwis.libs:openwis-libs:[unknown-version]: Could not find artifact io.openwis:openwis:pom:3.14.16-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 4, column 11 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ io.openwis.management.service:openwis-management-service-common:[unknown-version],/home/openwis/OpenWIS/openwis/openwis-management/openwis-management-service/openwis-management-service-common/pom.xml, line 48, column 12 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-war-plugin is missing. @ io.openwis.securityservice:openwis-securityservice-war:[unknown-version], /home/openwis/OpenWIS/openwis/openwis-securityservice/openwis-securityservice-war/pom.xml, line 89, column 21 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-war-plugin is missing. @ io.openwis.stagingpost:openwis-stagingpost:[unknown-version], /home/openwis/OpenWIS/openwis/openwis-stagingpost/pom.xml, line 32, column 21 [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-war-plugin is missing. @ line 174, column 21 [WARNING] 'build.plugins.plugin.version' for org.codehaus.mojo:exec-maven-plugin is missing. @ io.openwis.tools:load-test-data:[unknown-version], /home/openwis/OpenWIS/openwis/openwis-tools/load-test-data/pom.xml, line 109, column 23 @ [ERROR] The build could not read 1 project -> [Help 1] [ERROR] [ERROR] The project io.openwis.libs:openwis-libs:[unknown-version] (/home/openwis/OpenWIS/openwis/openwis-libs/pom.xml) has 1 error [ERROR] Non-resolvable parent POM for io.openwis.libs:openwis-libs:[unknown-version]: Could not find artifact io.openwis:openwis:pom:3.14.16-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 4, column 11 -> [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

I believe I have the most recent repo for this build:

commit 15f363e https://github.com/OpenWIS/openwis/commit/15f363e857e482694289ad0ecb37de6d32848812 (HEAD -> openwis-3.16, origin/openwis-3.16) Author: brahim @.***> Date: Mon Dec 12 14:16:04 2022 +0100

BHA: Admin portal create user bug fix

commit 5667b4b https://github.com/OpenWIS/openwis/commit/5667b4b2e765b21ad232da548aa451a2df4997ad Author: brahim @.***> Date: Fri Dec 9 15:26:48 2022 +0100

BHA: GenerateSPConfFiles.zip fix

commit 407d66c https://github.com/OpenWIS/openwis/commit/407d66c45f62d68d119383893de0debb89c145a6 Author: brahim @.***> Date: Wed Dec 7 15:04:31 2022 +0100

BHA: Add openwis-3.16 installion guide

commit bcee054 https://github.com/OpenWIS/openwis/commit/bcee0542e98c12d18afe5cd554a01215f1db9928 Author: brahim @.***> Date: Wed Dec 7 15:01:35 2022 +0100

BHA: add solr configuration file

— Reply to this email directly, view it on GitHub https://github.com/OpenWIS/openwis/issues/398#issuecomment-1371230565, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLJCV2NDF7OZ4YASHRZSPLWQWY7JANCNFSM53SHHNXQ . You are receiving this because you commented.Message ID: @.***>