search

minio / mint

Collection of tests to detect overall correctness of MinIO server.
Apache License 2.0
81 stars 50 forks source link

Add encryption tests using AWS S3 java SDK #264

Closed vadmeste closed 6 years ago

vadmeste commented 6 years ago

Fixes #262

vadmeste commented 6 years ago

Ready for review & merge

vadmeste commented 6 years ago

ping @nitisht & @balamurugana

nitisht commented 6 years ago

@vadmeste I tried building this locally, but the mint build failed. Am I missing something?

Here is the build trace

Step 22/50 : RUN build/aws-sdk-java/install.sh
 ---> Running in 2a3eb89039fa
Buildfile: /mint/build/aws-sdk-java/build.xml

download-ivy:
    [mkdir] Created dir: /root/.ant/lib
      [get] Getting: https://repo1.maven.org/maven2/org/apache/ivy/ivy/2.1.0-rc2/ivy-2.1.0-rc2.jar
      [get] To: /root/.ant/lib/ivy.jar

init-ivy:

BUILD SUCCESSFUL
Total time: 5 seconds
Buildfile: /mint/build/aws-sdk-java/build.xml

resolve:
[ivy:retrieve] :: Ivy 2.1.0-rc2 - 20090704004254 :: http://ant.apache.org/ivy/ ::
[ivy:retrieve] :: loading settings :: url = jar:file:/root/.ant/lib/ivy.jar!/org/apache/ivy/core/settings/ivysettings.xml
[ivy:retrieve] :: resolving dependencies :: org.apache#hello-ivy;working@2a3eb89039fa
[ivy:retrieve]  confs: [default]
[ivy:retrieve]  found com.amazonaws#aws-java-sdk-s3;1.11.289 in public
[ivy:retrieve]  found com.amazonaws#aws-java-sdk-kms;1.11.289 in public
[ivy:retrieve]  found com.amazonaws#aws-java-sdk-core;1.11.289 in public
[ivy:retrieve]  found software.amazon.ion#ion-java;1.0.2 in public
[ivy:retrieve]  found com.fasterxml.jackson.core#jackson-databind;2.6.7.1 in public
[ivy:retrieve]  found com.fasterxml.jackson.core#jackson-annotations;2.6.0 in public
[ivy:retrieve]  found com.fasterxml.jackson.core#jackson-core;2.6.7 in public
[ivy:retrieve]  found com.fasterxml.jackson.dataformat#jackson-dataformat-cbor;2.6.7 in public
[ivy:retrieve]  found joda-time#joda-time;2.8.1 in public
[ivy:retrieve]  found com.amazonaws#jmespath-java;1.11.289 in public
[ivy:retrieve] downloading http://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.289/aws-java-sdk-s3-1.11.289-sources.jar ...
[ivy:retrieve] .........................................................................................................
[ivy:retrieve] ...............................................................................................................................
[ivy:retrieve] ..................................................................................... (814kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] com.amazonaws#aws-java-sdk-s3;1.11.289!aws-java-sdk-s3.jar(source) (5265ms)
[ivy:retrieve] downloading http://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.289/aws-java-sdk-s3-1.11.289-javadoc.jar ...
[ivy:retrieve] ..............................................................................................................................
[ivy:retrieve] ....................................................................................................................................................................
[ivy:retrieve] ......................................................................................................................................................................................................................................................
[ivy:retrieve] ......................................................................................................................................................................................................................................................................................................................................................................................
[ivy:retrieve] ............................................................................................................................................................................................ (3121kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] com.amazonaws#aws-java-sdk-s3;1.11.289!aws-java-sdk-s3.jar(javadoc) (8029ms)
[ivy:retrieve] downloading http://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.289/aws-java-sdk-s3-1.11.289.jar ...
[ivy:retrieve] ............................................................................................................................................................................... (817kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] com.amazonaws#aws-java-sdk-s3;1.11.289!aws-java-sdk-s3.jar (2195ms)
[ivy:retrieve] downloading http://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.289/aws-java-sdk-kms-1.11.289.jar ...
[ivy:retrieve] ................................................................................................................ (394kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] com.amazonaws#aws-java-sdk-kms;1.11.289!aws-java-sdk-kms.jar (1667ms)
[ivy:retrieve] downloading http://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.289/aws-java-sdk-core-1.11.289.jar ...
[ivy:retrieve] ................................................................................................................................................................................................................................................................. (840kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] com.amazonaws#aws-java-sdk-core;1.11.289!aws-java-sdk-core.jar (2002ms)
[ivy:retrieve] downloading http://repo1.maven.org/maven2/com/amazonaws/jmespath-java/1.11.289/jmespath-java-1.11.289.jar ...
[ivy:retrieve] ....... (26kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] com.amazonaws#jmespath-java;1.11.289!jmespath-java.jar (1200ms)
[ivy:retrieve] downloading http://repo1.maven.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar ...
[ivy:retrieve] ................... (552kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] software.amazon.ion#ion-java;1.0.2!ion-java.jar(bundle) (2673ms)
[ivy:retrieve] downloading http://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.1/jackson-databind-2.6.7.1.jar ...
[ivy:retrieve] .....................................................................................................................................................................................................................
[ivy:retrieve] ................................................................................................................. (1138kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] com.fasterxml.jackson.core#jackson-databind;2.6.7.1!jackson-databind.jar (3069ms)
[ivy:retrieve] downloading http://repo1.maven.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.jar ...
[ivy:retrieve] ...... (47kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] com.fasterxml.jackson.dataformat#jackson-dataformat-cbor;2.6.7!jackson-dataformat-cbor.jar (1662ms)
[ivy:retrieve] downloading http://repo1.maven.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar ...
[ivy:retrieve] .................................................................................................................. (607kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] joda-time#joda-time;2.8.1!joda-time.jar (2005ms)
[ivy:retrieve] downloading http://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar ...
[ivy:retrieve] ................ (45kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] com.fasterxml.jackson.core#jackson-annotations;2.6.0!jackson-annotations.jar(bundle) (1923ms)
[ivy:retrieve] downloading http://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.jar ...
[ivy:retrieve] ............................................................. (252kB)
[ivy:retrieve] .. (0kB)
[ivy:retrieve]  [SUCCESSFUL ] com.fasterxml.jackson.core#jackson-core;2.6.7!jackson-core.jar(bundle) (1710ms)
[ivy:retrieve] :: resolution report :: resolve 45227ms :: artifacts dl 33448ms
    ---------------------------------------------------------------------
    |                  |            modules            ||   artifacts   |
    |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
    ---------------------------------------------------------------------
    |      default     |   12  |   10  |   10  |   0   ||   12  |   12  |
    ---------------------------------------------------------------------
[ivy:retrieve] 
[ivy:retrieve] :: problems summary ::
[ivy:retrieve] :::: WARNINGS
[ivy:retrieve]  io problem while parsing ivy file: http://repo1.maven.org/maven2/org/apache/commons/commons-parent/28/commons-parent-28.pom: Impossible to load parent for file:/root/.ant/cache/org.apache.commons/commons-parent/ivy-28.xml.original. Parent=org.apache#apache;13
[ivy:retrieve]  io problem while parsing ivy file: http://repo1.maven.org/maven2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.pom: Impossible to load parent for file:/root/.ant/cache/commons-logging/commons-logging/ivy-1.1.3.xml.original. Parent=org.apache.commons#commons-parent;28
[ivy:retrieve]      module not found: commons-logging#commons-logging;1.1.3
[ivy:retrieve]  ==== local: tried
[ivy:retrieve]    /root/.ant/local/org.apache/apache/13/ivys/ivy.xml
[ivy:retrieve]    -- artifact org.apache#apache;13!apache.jar:
[ivy:retrieve]    /root/.ant/local/org.apache/apache/13/jars/apache.jar
[ivy:retrieve]  ==== shared: tried
[ivy:retrieve]    /root/.ant/shared/org.apache/apache/13/ivys/ivy.xml
[ivy:retrieve]    -- artifact org.apache#apache;13!apache.jar:
[ivy:retrieve]    /root/.ant/shared/org.apache/apache/13/jars/apache.jar
[ivy:retrieve]  ==== public: tried
[ivy:retrieve]    http://repo1.maven.org/maven2/org/apache/apache/13/apache-13.pom
[ivy:retrieve]    -- artifact org.apache#apache;13!apache.jar:
[ivy:retrieve]    http://repo1.maven.org/maven2/org/apache/apache/13/apache-13.jar
[ivy:retrieve]  io problem while parsing ivy file: http://repo1.maven.org/maven2/org/apache/httpcomponents/project/7/project-7.pom: Impossible to load parent for file:/root/.ant/cache/org.apache.httpcomponents/project/ivy-7.xml.original. Parent=org.apache#apache;13
[ivy:retrieve]  io problem while parsing ivy file: http://repo1.maven.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.2/httpcomponents-client-4.5.2.pom: Impossible to load parent for file:/root/.ant/cache/org.apache.httpcomponents/httpcomponents-client/ivy-4.5.2.xml.original. Parent=org.apache.httpcomponents#project;7
[ivy:retrieve]  io problem while parsing ivy file: http://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.pom: Impossible to load parent for file:/root/.ant/cache/org.apache.httpcomponents/httpclient/ivy-4.5.2.xml.original. Parent=org.apache.httpcomponents#httpcomponents-client;4.5.2
[ivy:retrieve]      module not found: org.apache.httpcomponents#httpclient;4.5.2
[ivy:retrieve]  ==== local: tried
[ivy:retrieve]    /root/.ant/local/org.apache/apache/13/ivys/ivy.xml
[ivy:retrieve]    -- artifact org.apache#apache;13!apache.jar:
[ivy:retrieve]    /root/.ant/local/org.apache/apache/13/jars/apache.jar
[ivy:retrieve]  ==== shared: tried
[ivy:retrieve]    /root/.ant/shared/org.apache/apache/13/ivys/ivy.xml
[ivy:retrieve]    -- artifact org.apache#apache;13!apache.jar:
[ivy:retrieve]    /root/.ant/shared/org.apache/apache/13/jars/apache.jar
[ivy:retrieve]  ==== public: tried
[ivy:retrieve]    http://repo1.maven.org/maven2/org/apache/apache/13/apache-13.pom
[ivy:retrieve]    -- artifact org.apache#apache;13!apache.jar:
[ivy:retrieve]    http://repo1.maven.org/maven2/org/apache/apache/13/apache-13.jar
[ivy:retrieve]      ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:retrieve]      ::          UNRESOLVED DEPENDENCIES         ::
[ivy:retrieve]      ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:retrieve]      :: commons-logging#commons-logging;1.1.3: not found
[ivy:retrieve]      :: org.apache.httpcomponents#httpclient;4.5.2: not found
[ivy:retrieve]      ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:retrieve] 
[ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

BUILD FAILED
/mint/build/aws-sdk-java/build.xml:25: impossible to resolve dependencies:
    resolve failed - see output for details

Total time: 1 minute 19 seconds
cp: cannot stat 'build/jar/FunctionalTests.jar': No such file or directory
The command '/bin/sh -c build/aws-sdk-java/install.sh' returned a non-zero code: 1
vadmeste commented 6 years ago

@nitisht this is mostly a one time error or network problem.. can you try one time again ?

nitisht commented 6 years ago

@nitisht this is mostly a one time error or network problem.. can you try one time again ?

You're right, looks like building now

nitisht commented 6 years ago

tested locally and works fine @vadmeste . Can you update the Dockerfile / Readme.md as well?

vadmeste commented 6 years ago

@nitisht oh yes, Readme needs to be updated but Dockerfile.dev is already updated, do we need to do something with Dockerfile ?

nitisht commented 6 years ago

do we need to do something with Dockerfile ?

Dockerfile automatically loops through all the directories, so nothing to be done there.

vadmeste commented 6 years ago

ping @balamurugana, please take a look

vadmeste commented 6 years ago

@balamurugana I removed RandAccessFile & InputStreamNopCloser classes so it is easier to understand. Please review again.