search

keeps / dbptk-developer

DBPTK Developer - library and command-line tool for execution of database preservation actions
http://www.database-preservation.com
GNU Lesser General Public License v3.0
43 stars 19 forks source link

The "master" branch does not build from a clean checkout #319

Closed mhvelplund closed 7 years ago

mhvelplund commented 7 years ago

Description:

Steps required to reproduce the bug:

  1. Remove local package cache: rm -rf ~/.m2/repository
  2. Remove any extra repositories from settings (~/.m2/settings.xml)
  3. Run Maven: mvn clean package

The result is the following:

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] Database Preservation Toolkit
[INFO] dbptk-model
[INFO] dbptk-modules
[INFO] dbptk-module-dbml
[INFO] dbptk-module-jdbc
[INFO] dbptk-module-list-tables
[INFO] dbptk-module-ms-access
[INFO] dbptk-module-mysql
[INFO] dbptk-module-oracle
[INFO] dbptk-module-postgresql
[INFO] dbptk-module-siard
[INFO] dbptk-module-solr
[INFO] dbptk-module-sql-server
[INFO] dbptk-core
[INFO] dbptk-module-db2
[INFO] dbptk-module-odbc
[INFO] dbptk-module-sql-file
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Database Preservation Toolkit 2.0.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ dbptk ---
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building dbptk-model 2.0.0
[INFO] ------------------------------------------------------------------------
Downloading: http://artifactory.keep.pt/keep/jspf/core/jspf.core/1.0.2/jspf.core-1.0.2.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Database Preservation Toolkit ...................... SUCCESS [  0.097 s]
[INFO] dbptk-model ........................................ FAILURE [  0.660 s]
[INFO] dbptk-modules ...................................... SKIPPED
[INFO] dbptk-module-dbml .................................. SKIPPED
[INFO] dbptk-module-jdbc .................................. SKIPPED
[INFO] dbptk-module-list-tables ........................... SKIPPED
[INFO] dbptk-module-ms-access ............................. SKIPPED
[INFO] dbptk-module-mysql ................................. SKIPPED
[INFO] dbptk-module-oracle ................................ SKIPPED
[INFO] dbptk-module-postgresql ............................ SKIPPED
[INFO] dbptk-module-siard ................................. SKIPPED
[INFO] dbptk-module-solr .................................. SKIPPED
[INFO] dbptk-module-sql-server ............................ SKIPPED
[INFO] dbptk-core ......................................... SKIPPED
[INFO] dbptk-module-db2 ................................... SKIPPED
[INFO] dbptk-module-odbc .................................. SKIPPED
[INFO] dbptk-module-sql-file .............................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.048 s
[INFO] Finished at: 2017-09-06T13:34:09+02:00
[INFO] Final Memory: 13M/299M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project dbptk-model: Could not resolve dependencies for project com.databasepreservation:dbptk-model:jar:2.0.0: Failed to collect dependencies at jspf.core:jspf.core:jar:1.0.2: Failed to read artifact descriptor for jspf.core:jspf.core:jar:1.0.2: Could not transfer artifact jspf.core:jspf.core:pom:1.0.2 from/to KEEPS-Artifacts (http://artifactory.keep.pt/keep): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :dbptk-model
mhvelplund commented 7 years ago

It's trivial to add a repository to the top-level POM to get past downloading jspf.core-1.0.2.jar, e.g.:

<repository>
  <id>KEEPS-Artifacts-Alt</id>
  <name>KEEP Artifacts-releases (alternative)</name>
  <url>http://repository.mygrid.org.uk/artifactory/simple/keep</url>
</repository>

... but that merely pushes the compiler failure to later in the process:

...
[INFO] ------------------------------------------------------------------------
[INFO] Building dbptk-module-siard 2.0.0
[INFO] ------------------------------------------------------------------------
Downloading: http://artifactory.keep.pt/keep/com/databasepreservation/dbptk-bindings-siard1/1.1.1/dbptk-bindings-siard1-1.1.1.pom
Downloading: http://artifactory.keep.pt/keep/com/databasepreservation/dbptk-bindings-siard2/1.1.1/dbptk-bindings-siard2-1.1.1.pom
Downloading: http://artifactory.keep.pt/keep/com/databasepreservation/dbptk-bindings-siarddk/1.3.0/dbptk-bindings-siarddk-1.3.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Database Preservation Toolkit ...................... SUCCESS [  0.096 s]
[INFO] dbptk-model ........................................ SUCCESS [  2.191 s]
[INFO] dbptk-modules ...................................... SUCCESS [  0.139 s]
[INFO] dbptk-module-dbml .................................. SUCCESS [  0.154 s]
[INFO] dbptk-module-jdbc .................................. SUCCESS [  0.206 s]
[INFO] dbptk-module-list-tables ........................... SUCCESS [  0.080 s]
[INFO] dbptk-module-ms-access ............................. SUCCESS [  0.095 s]
[INFO] dbptk-module-mysql ................................. SUCCESS [  0.091 s]
[INFO] dbptk-module-oracle ................................ SUCCESS [  0.075 s]
[INFO] dbptk-module-postgresql ............................ SUCCESS [  0.110 s]
[INFO] dbptk-module-siard ................................. FAILURE [  1.137 s]
[INFO] dbptk-module-solr .................................. SKIPPED
[INFO] dbptk-module-sql-server ............................ SKIPPED
[INFO] dbptk-core ......................................... SKIPPED
[INFO] dbptk-module-db2 ................................... SKIPPED
[INFO] dbptk-module-odbc .................................. SKIPPED
[INFO] dbptk-module-sql-file .............................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.669 s
[INFO] Finished at: 2017-09-06T13:45:40+02:00
[INFO] Final Memory: 37M/484M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project dbptk-module-siard: Could not resolve dependencies for project com.databasepreservation:dbptk-module-siard:jar:2.0.0: Failed to collect dependencies at com.databasepreservation:dbptk-bindings-siard1:jar:1.1.1: Failed to read artifact descriptor for com.databasepreservation:dbptk-bindings-siard1:jar:1.1.1: Could not transfer artifact com.databasepreservation:dbptk-bindings-siard1:pom:1.1.1 from/to KEEPS-Artifacts (http://artifactory.keep.pt/keep): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target -> [Help 1]
luis100 commented 7 years ago

This happens because the repository http://artifactory.keep.pt/keep now redirects to its HTTPS version (https://artifactory.keep.pt/keep) which has a LetsEncrypt certificate and you are using an outdated version of Java in your maven command.

Please update your Java to a version equal or greater to Java 8 update 101. See this for more details: https://stackoverflow.com/a/34111150

To check which Java version you are using you can do:

$ mvn -version
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T20:39:06+01:00)
Maven home: /usr/local/Cellar/maven/3.5.0/libexec
Java version: 1.8.0_144, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre
Default locale: pt_PT, platform encoding: UTF-8
OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "mac"
mhvelplund commented 7 years ago

Your were right. I was using an older version og Java 1.8. Summarizing the SO for others coming here, I had to do the following to get past the problem (on OSX):

$ export JAVA_HOME=$(/usr/libexec/java_home)
$ sudo cp -a $JAVA_HOME/jre/lib/security/cacerts $JAVA_HOME/jre/lib/security/cacerts.orig
$ wget https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.der
$ sudo keytool -trustcacerts -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass changeit -noprompt -importcert -alias lets-encrypt-x3-cross-signed -file lets-encrypt-x3-cross-signed.der

Compilation now fails later in the dbptk-core module while running tests:

Failed tests:
  TestSIARDDKImportModule.testArchiveSplitInMultipleFolders:37 Expected the content of [/var/folders/2_/10gf5rhs4bb9qd5sfvkdf9xh0000gn/T/AVID.TST.4000.1/Indices/fileIndex.xml] to match the content of [/Users/mhv/tmp/db-preservation-toolkit/dbptk-core/target/test-classes/siarddk/AVID.TST.4000.1/Indices/fileIndex.xml]
  TestSIARDDKImportModule.testArchiveWithXmlHexBinaryData:67 Expected import of siard-dk archive [/Users/mhv/tmp/db-preservation-toolkit/dbptk-core/target/test-classes/siarddk/AVID.HEX.1000.1] followed by export to siard-dk archive [/var/folders/2_/10gf5rhs4bb9qd5sfvkdf9xh0000gn/T/AVID.HEX.2000.1] to succeed.

That can be handled with: mvn clean package -DskipTests 😄

luis100 commented 7 years ago

Unit tests might be failing due to missing installed dependencies. @chalkos please create another issue to document which dependencies need to be installed in order to run the full set of unit tests.

luis100 commented 7 years ago

@mhvelplund to note that simply downloading and installing the latest Java JDK would also have worked.

mhvelplund commented 7 years ago

The POM files indicate that the project is a Java 7 project? I don't think there is a version of JDK 7 that has the newest certs.

luis100 commented 7 years ago

Yes, that is true, if you'd like to use Java 7 to compile it your solutions is the most adequate.

mhvelplund commented 7 years ago

"adequate" 🤣

Anyway, thanks for the quick response!

chalkos commented 7 years ago

please create another issue to document which dependencies need to be installed in order to run the full set of unit tests.

@luis100 I don't think there are any besides having databases configured as described in the README. But the development documentation could use an update. Added an issue for that: #320