Apache Thrift 0.12.0 is required. Building it reports unsupported .NET, etc. Installing 0.13.0 using yum results in an error on mvn package.

Reporter: Lutz Weischer

_{Note: This issue was originally created as PARQUET-2000. Please see the migration documentation for further details.}

Gabor Szadovszky / @gszadovszky: The current master release requires 0.13.0. The build would fail with any other version of the thrift binary. Could you please explain what mvn error do you see when you use 0.13.0?

Lutz Weischer: [jw@cn06 parquet]$ mvn package [INFO] Scanning for projects... [INFO] [INFO] -----------------< org.apache.parquet:parquet-format >------------------ [INFO] Building Apache Parquet Format 2.9.0-SNAPSHOT [INFO] -------------------------------[ jar ]-------------------------------- [INFO] [INFO] — thrift-maven-plugin:0.10.0:compile (thrift-sources) @ parquet-format — [INFO] [INFO] — exec-maven-plugin:1.2.1:exec (check-thrift-version) @ parquet-format —

========== [FATAL] Build is configured to require Thrift version 0.12.0 ========== ========== Currently installed: Thrift version 0.13.0

[INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.682 s [INFO] Finished at: 2021-03-17T08:54:58+01:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (check-thrift-version) on project parquet-format: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [jw@cn06 parquet]$ thrift -version Thrift version 0.13.0 [jw@cn06 parquet]$ mvn install -DskipTests [INFO] Scanning for projects... [INFO] [INFO] -----------------< org.apache.parquet:parquet-format >------------------ [INFO] Building Apache Parquet Format 2.9.0-SNAPSHOT [INFO] -------------------------------[ jar ]-------------------------------- [INFO] [INFO] — thrift-maven-plugin:0.10.0:compile (thrift-sources) @ parquet-format — [INFO] [INFO] — exec-maven-plugin:1.2.1:exec (check-thrift-version) @ parquet-format —

========== [FATAL] Build is configured to require Thrift version 0.12.0 ========== ========== Currently installed: Thrift version 0.13.0

[INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.428 s [INFO] Finished at: 2021-03-17T08:56:36+01:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (check-thrift-version) on project parquet-format: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [jw@cn06 parquet]$

Gabor Szadovszky / @gszadovszky: So, you are trying to build parquet-format (and not parquet-mr). parquet-format still configured for thrift 0.12.0 while it does not really means anything because the parquet-format thrift compile is only for testing purposes. The production code for parquet-format is generated inside parquet-mr (in the module parquet-format-structures).

Why are you trying to build parquet-format? If you are interested in the java implementation of parquet I would suggest looking into parquet-mr.

Lutz Weischer: Switched to parquet-mr.

[jw@cn05 parquet]$ LC_ALL=C mvn install ... [INFO] [INFO] — maven-jar-plugin:2.4:jar (default-jar) @ parquet-hadoop — [INFO] Building jar: /home/jw/apache/parquet/parquet-hadoop/target/parquet-hadoop-1.12.0-SNAPSHOT.jar [INFO] [INFO] — maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ parquet-hadoop — [INFO] [INFO] — maven-jar-plugin:2.4:test-jar (default) @ parquet-hadoop — [INFO] Building jar: /home/jw/apache/parquet/parquet-hadoop/target/parquet-hadoop-1.12.0-SNAPSHOT-tests.jar [INFO] [INFO] — maven-shade-plugin:3.2.1:shade (default) @ parquet-hadoop — [INFO] Excluding org.apache.parquet:parquet-column:jar:1.12.0-SNAPSHOT from the shaded jar. [INFO] Excluding org.apache.parquet:parquet-common:jar:1.12.0-SNAPSHOT from the shaded jar. [INFO] Excluding org.apache.yetus:audience-annotations:jar:0.12.0 from the shaded jar. [INFO] Excluding org.apache.parquet:parquet-encoding:jar:1.12.0-SNAPSHOT from the shaded jar. [INFO] Including net.openhft:zero-allocation-hashing:jar:0.9 in the shaded jar. [INFO] Excluding org.apache.parquet:parquet-format-structures:jar:1.12.0-SNAPSHOT from the shaded jar. [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.22 from the shaded jar. [INFO] Excluding javax.annotation:javax.annotation-api:jar:1.3.2 from the shaded jar. [INFO] Excluding org.apache.parquet:parquet-jackson:jar:1.12.0-SNAPSHOT from the shaded jar. [INFO] Including com.fasterxml.jackson.core:jackson-core:jar:2.12.2 in the shaded jar. [INFO] Including com.fasterxml.jackson.core:jackson-databind:jar:2.12.2 in the shaded jar. [INFO] Including com.fasterxml.jackson.core:jackson-annotations:jar:2.12.2 in the shaded jar. [INFO] Excluding org.xerial.snappy:snappy-java:jar:1.1.8 from the shaded jar. [INFO] Excluding commons-pool:commons-pool:jar:1.6 from the shaded jar. [INFO] Including it.unimi.dsi:fastutil:jar:8.4.2 in the shaded jar. [INFO] Excluding com.github.luben:zstd-jni:jar:1.4.9-1 from the shaded jar. [INFO] Minimizing jar org.apache.parquet:parquet-hadoop:jar:1.12.0-SNAPSHOT [INFO] Minimized 11128 -> 491 (4%) [INFO] Replacing original artifact with shaded artifact. [INFO] Replacing /home/jw/apache/parquet/parquet-hadoop/target/parquet-hadoop-1.12.0-SNAPSHOT.jar with /home/jw/apache/parquet/parquet-hadoop/target/parquet-hadoop-1.12.0-SNAPSHOT-shaded.jar [INFO] Dependency-reduced POM written at: /home/jw/apache/parquet/parquet-hadoop/dependency-reduced-pom.xml [INFO] [INFO] — maven-failsafe-plugin:2.10:integration-test (default) @ parquet-hadoop — [INFO] Failsafe report directory: /home/jw/apache/parquet/parquet-hadoop/target/failsafe-reports

T E S T S

Running org.apache.parquet.hadoop.ITTestEncryptionOptions [main] INFO org.apache.parquet.hadoop.TestEncryptionOptions - ======== testInteropReadEncryptedParquetFiles target/parquet-testing/data ======== [main] INFO org.apache.parquet.hadoop.TestEncryptionOptions - Download interop files if needed WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.apache.hadoop.security.authentication.util.KerberosUtil (file:/home/jw/.m2/repository/org/apache/hadoop/hadoop-auth/2.10.1/hadoop-auth-2.10.1.jar) to method sun.security.krb5.Config.getInstance() WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.security.authentication.util.KerberosUtil WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 10:20:52,298 DEBUG Tracer: 106 - sampler.classes = ; loaded no samplers 10:20:52,318 DEBUG Tracer: 128 - span.receiver.classes = ; loaded no span receivers [main] INFO org.apache.parquet.hadoop.TestEncryptionOptions - target/parquet-testing/data exists?: true [main] INFO org.apache.parquet.hadoop.TestEncryptionOptions - Download interop file: https://github.com/apache/parquet-testing/raw/40379b3/data/uniform_encryption.parquet.encrypted Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 8.554 sec <<< FAILURE! Running org.apache.parquet.encodings.FileEncodingsIT Testing codecs: [UNCOMPRESSED] Testing PARQUET_1_0/BOOLEAN/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/BOOLEAN/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/BOOLEAN/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/BOOLEAN/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/INT32/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/INT32/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/INT32/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/INT32/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/INT64/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/INT64/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/INT64/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/INT64/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/INT96/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/INT96/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/INT96/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/INT96/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/FLOAT/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/FLOAT/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/FLOAT/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/FLOAT/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/DOUBLE/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/DOUBLE/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/DOUBLE/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/DOUBLE/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/BINARY/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/BINARY/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/BINARY/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/BINARY/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/FIXED_LEN_BYTE_ARRAY/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/FIXED_LEN_BYTE_ARRAY/UNCOMPRESSED encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_1_0/FIXED_LEN_BYTE_ARRAY/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/FIXED_LEN_BYTE_ARRAY/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,108.336 sec

Results :

Tests in error: testInteropReadEncryptedParquetFiles(org.apache.parquet.hadoop.ITTestEncryptionOptions): github.com: Name or service not known

Tests run: 17, Failures: 0, Errors: 1, Skipped: 0

[INFO] [INFO] — maven-failsafe-plugin:2.10:verify (default) @ parquet-hadoop — [INFO] Failsafe report directory: /home/jw/apache/parquet/parquet-hadoop/target/failsafe-reports [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for Apache Parquet MR 1.12.0-SNAPSHOT: [INFO] [INFO] Apache Parquet MR .................................. SUCCESS [ 10.510 s] [INFO] Apache Parquet Format Structures ................... SUCCESS [ 9.290 s] [INFO] Apache Parquet Generator ........................... SUCCESS [ 3.472 s] [INFO] Apache Parquet Common .............................. SUCCESS [ 10.361 s] [INFO] Apache Parquet Encodings ........................... SUCCESS [ 37.394 s] [INFO] Apache Parquet Column .............................. SUCCESS [01:41 min] [INFO] Apache Parquet Arrow ............................... SUCCESS [ 3.497 s] [INFO] Apache Parquet Jackson ............................. SUCCESS [ 5.068 s] [INFO] Apache Parquet Hadoop .............................. FAILURE [23:20 min] [INFO] Apache Parquet Avro ................................ SKIPPED [INFO] Apache Parquet Benchmarks .......................... SKIPPED [INFO] Apache Parquet Pig ................................. SKIPPED [INFO] Apache Parquet Thrift .............................. SKIPPED [INFO] Apache Parquet Cascading [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Cascading (for Cascading 3.0 onwards) [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Command-line ........................ SKIPPED [INFO] Apache Parquet Pig Bundle .......................... SKIPPED [INFO] Apache Parquet Protobuf ............................ SKIPPED [INFO] Apache Parquet Scala ............................... SKIPPED [INFO] Apache Parquet Scrooge [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Hadoop Bundle ....................... SKIPPED [INFO] Apache Parquet Tools; [Deprecated, will be removed after 1.12; Use parquet-cli instead] SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 26:21 min [INFO] Finished at: 2021-03-22T10:39:29+01:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.10:verify (default) on project parquet-hadoop: There are test failures. [ERROR] [ERROR] Please refer to /home/jw/apache/parquet/parquet-hadoop/target/failsafe-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :parquet-hadoop [jw@cn05 parquet]$

Gabor Szadovszky / @gszadovszky: The failing test tries to download a file from github directly. The exact link is listed in the log. It seems github.com is not accessible from your environment.

Lutz Weischer: I restarted from scratch and get this:

LC_ALL=C mvn install ... Testing PARQUET_1_0/FIXED_LEN_BYTE_ARRAY/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Testing PARQUET_2_0/FIXED_LEN_BYTE_ARRAY/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,100.501 sec

Results :

Tests in error: testInteropReadEncryptedParquetFiles(org.apache.parquet.hadoop.ITTestEncryptionOptions): github.com: Name or service not known

Tests run: 17, Failures: 0, Errors: 1, Skipped: 0

[INFO] [INFO] — maven-failsafe-plugin:2.10:verify (default) @ parquet-hadoop — [INFO] Failsafe report directory: /data/jw/parquet/parquet-hadoop/target/failsafe-reports [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for Apache Parquet MR 1.12.0-SNAPSHOT: [INFO] [INFO] Apache Parquet MR .................................. SUCCESS [ 8.688 s] [INFO] Apache Parquet Format Structures ................... SUCCESS [ 8.891 s] [INFO] Apache Parquet Generator ........................... SUCCESS [ 3.289 s] [INFO] Apache Parquet Common .............................. SUCCESS [ 10.041 s] [INFO] Apache Parquet Encodings ........................... SUCCESS [ 33.835 s] [INFO] Apache Parquet Column .............................. SUCCESS [01:32 min] [INFO] Apache Parquet Arrow ............................... SUCCESS [ 3.288 s] [INFO] Apache Parquet Jackson ............................. SUCCESS [ 4.596 s] [INFO] Apache Parquet Hadoop .............................. FAILURE [22:59 min] [INFO] Apache Parquet Avro ................................ SKIPPED [INFO] Apache Parquet Benchmarks .......................... SKIPPED [INFO] Apache Parquet Pig ................................. SKIPPED [INFO] Apache Parquet Thrift .............................. SKIPPED [INFO] Apache Parquet Cascading [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Cascading (for Cascading 3.0 onwards) [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Command-line ........................ SKIPPED [INFO] Apache Parquet Pig Bundle .......................... SKIPPED [INFO] Apache Parquet Protobuf ............................ SKIPPED [INFO] Apache Parquet Scala ............................... SKIPPED [INFO] Apache Parquet Scrooge [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Hadoop Bundle ....................... SKIPPED [INFO] Apache Parquet Tools; [Deprecated, will be removed after 1.12; Use parquet-cli instead] SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 25:45 min [INFO] Finished at: 2021-03-24T15:06:55+01:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.10:verify (default) on project parquet-hadoop: There are test failures. [ERROR] [ERROR] Please refer to /data/jw/parquet/parquet-hadoop/target/failsafe-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :parquet-hadoop (base) [jw@cn06 parquet]$

I can access github.com, because I git clone'd from it.

(base) [jw@cn06 parquet]$ ls /data/jw/parquet/parquet-hadoop/target/failsafe-reports TEST-org.apache.parquet.encodings.FileEncodingsIT.xml org.apache.parquet.encodings.FileEncodingsIT.txt TEST-org.apache.parquet.hadoop.ITTestEncryptionOptions.xml org.apache.parquet.hadoop.ITTestEncryptionOptions.txt failsafe-summary.xml (base) [jw@cn06 parquet]$ nano /data/jw/parquet/parquet-hadoop/target/failsafe-reports/failsafe-summary.xml

shows

<?xml version="1.0" encoding="UTF-8"?>

Gabor Szadovszky / @gszadovszky: Could you try to get the file https://github.com/apache/parquet-testing/raw/40379b3/data/encrypt_columns_and_footer.parquet.encrypted manually (e.g. with wget or curl)? It would also help if you could dig up the whole stack trace from failsafe-reports.

@andersonm-ibm, do you have an idea why it is happening and why we get the error message github.com: Name or service not known?

Maya Anderson / @andersonm-ibm: @gszadovszky, just like you suggested, I expect the same error when trying to get the file [https://github.com/apache/parquet-testing/raw/40379b3/data/encrypt_columns_and_footer.parquet.encrypted] manually (e.g. with wget or curl), since this looks like a DNS resolution issue. [~LutzWeischerFujitsu] , to isolate the problem - does adding github.com directly to /etc/hosts make any difference?

Lutz Weischer: I got a fresh installation and have now this result:

(base) [jw@cn05 parquet]$ LC_ALL=C mvn install -DskipTests ... [INFO] Apache Parquet Pig ................................. SUCCESS [ 9.340 s] [INFO] Apache Parquet Thrift .............................. SUCCESS [ 11.328 s] [INFO] Apache Parquet Cascading [Deprecated, will be removed after 1.12] FAILURE [ 1.496 s] [INFO] Apache Parquet Cascading (for Cascading 3.0 onwards) [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Command-line ........................ SKIPPED [INFO] Apache Parquet Pig Bundle .......................... SKIPPED [INFO] Apache Parquet Protobuf ............................ SKIPPED [INFO] Apache Parquet Scala ............................... SKIPPED [INFO] Apache Parquet Scrooge [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Hadoop Bundle ....................... SKIPPED [INFO] Apache Parquet Tools; [Deprecated, will be removed after 1.12; Use parquet-cli instead] SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:21 min [INFO] Finished at: 2021-03-26T14:08:18+01:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal com.github.siom79.japicmp:japicmp-maven-plugin:0.14.2:cmp (default) on project parquet-cascading-deprecated: Could not resolve org.apache.parquet:parquet-cascading:jar:1.12.0 -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :parquet-cascading-deprecated (base) [jw@cn05 parquet]$

Trying differently:

(base) [jw@cn05 parquet]$ mvn install -DskipTests ... [INFO] Apache Parquet Pig ................................. SUCCESS [ 8.357 s] [INFO] Apache Parquet Thrift .............................. SUCCESS [ 10.033 s] [INFO] Apache Parquet Cascading [Deprecated, will be removed after 1.12] FAILURE [ 0.686 s] [INFO] Apache Parquet Cascading (for Cascading 3.0 onwards) [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Command-line ........................ SKIPPED [INFO] Apache Parquet Pig Bundle .......................... SKIPPED [INFO] Apache Parquet Protobuf ............................ SKIPPED [INFO] Apache Parquet Scala ............................... SKIPPED [INFO] Apache Parquet Scrooge [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Hadoop Bundle ....................... SKIPPED [INFO] Apache Parquet Tools; [Deprecated, will be removed after 1.12; Use parquet-cli instead] SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:06 min [INFO] Finished at: 2021-03-26T14:33:17+01:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.12:check (default) on project parquet-cascading-deprecated: Too many files with unapproved license: 1 See RAT report in: /data/jw/apache/metron/parquet/parquet-cascading-deprecated/target/rat.txt -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :parquet-cascading-deprecated (base) [jw@cn05 parquet]$

And:

(base) [jw@cn05 parquet]$ mvn install -DskipTests -Drat.skip=true ... [INFO] Apache Parquet Pig ................................. SUCCESS [ 8.106 s] [INFO] Apache Parquet Thrift .............................. SUCCESS [ 9.405 s] [INFO] Apache Parquet Cascading [Deprecated, will be removed after 1.12] FAILURE [ 0.566 s] [INFO] Apache Parquet Cascading (for Cascading 3.0 onwards) [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Command-line ........................ SKIPPED [INFO] Apache Parquet Pig Bundle .......................... SKIPPED [INFO] Apache Parquet Protobuf ............................ SKIPPED [INFO] Apache Parquet Scala ............................... SKIPPED [INFO] Apache Parquet Scrooge [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Hadoop Bundle ....................... SKIPPED [INFO] Apache Parquet Tools; [Deprecated, will be removed after 1.12; Use parquet-cli instead] SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:03 min [INFO] Finished at: 2021-03-26T14:42:35+01:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal com.github.siom79.japicmp:japicmp-maven-plugin:0.14.2:cmp (default) on project parquet-cascading-deprecated: Could not resolve org.apache.parquet:parquet-cascading:jar:1.12.0 -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :parquet-cascading-deprecated (base) [jw@cn05 parquet]$

Gabor Szadovszky / @gszadovszky: [~LutzWeischerFujitsu], this is a known issue. See PARQUET-2010 for details. You may either cherry-pick my change in the related PR or step back to the previous release in the history to go forward. I hope my change will get in soon so everyone would be able to continue working on the new release.

Lutz Weischer: Please be so kind to inform when the new (master) release is available and should not have that issue any more. I will than re-try. Thanks.

Gabor Szadovszky / @gszadovszky: [~LutzWeischerFujitsu], PARQUET-2010 is already in master so it should work now.

Lutz Weischer: Looks like we've progress but we're not there yet, due to failing tests. Using Java 11, setting ANT_OPTS with proxy:

(base) [jw@cn06 parquet]$ LC_ALL=C mvn install ... Testing PARQUET_2_0/FIXED_LEN_BYTE_ARRAY/UNCOMPRESSED + DICTIONARY encodings using ROW_GROUP_SIZE=131072 PAGE_SIZE=16384 Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,062.679 sec

Results :

Tests in error: testInteropReadEncryptedParquetFiles(org.apache.parquet.hadoop.ITTestEncryptionOptions): github.com: Name or service not known

Tests run: 17, Failures: 0, Errors: 1, Skipped: 0

[INFO] [INFO] — maven-failsafe-plugin:2.10:verify (default) @ parquet-hadoop — [INFO] Failsafe report directory: /data/jw/apache/parquet/parquet-hadoop/target/failsafe-reports [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for Apache Parquet MR 1.13.0-SNAPSHOT: [INFO] [INFO] Apache Parquet MR .................................. SUCCESS [ 8.110 s] [INFO] Apache Parquet Format Structures ................... SUCCESS [ 8.766 s] [INFO] Apache Parquet Generator ........................... SUCCESS [ 3.380 s] [INFO] Apache Parquet Common .............................. SUCCESS [ 9.466 s] [INFO] Apache Parquet Encodings ........................... SUCCESS [ 35.689 s] [INFO] Apache Parquet Column .............................. SUCCESS [01:33 min] [INFO] Apache Parquet Arrow ............................... SUCCESS [ 3.252 s] [INFO] Apache Parquet Jackson ............................. SUCCESS [ 4.634 s] [INFO] Apache Parquet Hadoop .............................. FAILURE [22:20 min] [INFO] Apache Parquet Avro ................................ SKIPPED [INFO] Apache Parquet Benchmarks .......................... SKIPPED [INFO] Apache Parquet Pig ................................. SKIPPED [INFO] Apache Parquet Thrift .............................. SKIPPED [INFO] Apache Parquet Cascading [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Cascading (for Cascading 3.0 onwards) [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Command-line ........................ SKIPPED [INFO] Apache Parquet Pig Bundle .......................... SKIPPED [INFO] Apache Parquet Protobuf ............................ SKIPPED [INFO] Apache Parquet Scala ............................... SKIPPED [INFO] Apache Parquet Scrooge [Deprecated, will be removed after 1.12] SKIPPED [INFO] Apache Parquet Hadoop Bundle ....................... SKIPPED [INFO] Apache Parquet Tools; [Deprecated, will be removed after 1.12; Use parquet-cli instead] SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 25:08 min [INFO] Finished at: 2021-04-01T12:04:30+02:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.10:verify (default) on project parquet-hadoop: There are test failures. [ERROR] [ERROR] Please refer to /data/jw/apache/parquet/parquet-hadoop/target/failsafe-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :parquet-hadoop (base) [jw@cn06 parquet]$

Gabor Szadovszky / @gszadovszky: [~LutzWeischerFujitsu], this is the previous issue you've already posted. Please, check my [previous comment|#comment-17307872].

Lutz Weischer: wget https://github.com/apache/parquet-testing/raw/40379b3/data/encrypt_columns_and_footer.parquet.encrypted

works.

LC_ALL=C mvn clean install

which includes the tests, works, using Java 16.

apache / parquet-java

build failed on AArch64, Fedora 33 #2587

========== [FATAL] Build is configured to require Thrift version 0.12.0 ========== ========== Currently installed: Thrift version 0.13.0

========== [FATAL] Build is configured to require Thrift version 0.12.0 ========== ========== Currently installed: Thrift version 0.13.0

T E S T S