qubole / rubix

Cache File System optimized for columnar formats and object stores
Apache License 2.0
183 stars 74 forks source link

new: dev: increase coverage for rubix-presto module #138

Closed jordanw-bq closed 4 years ago

jordanw-bq commented 6 years ago

Only test classes have been introduced, so testing on a Qubole cluster should not be required.

jordanw-bq commented 6 years ago

The Travis build is failing when trying to initialize the CachingPrestoS3FileSystem object. Seems to be an issue with a Java version mismatch (52.0 = Java 8). Could this be due to Rubix being compiled with Java 7, and Presto with Java 8?

java.lang.UnsupportedClassVersionError: com/facebook/presto/hive/PrestoS3FileSystem : Unsupported major.minor version 52.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:803)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:442)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:64)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:354)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:348)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:347)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:278)
    at sun.reflect.generics.factory.CoreReflectionFactory.makeNamedType(CoreReflectionFactory.java:114)
    at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:125)
    at sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49)
    at sun.reflect.generics.visitor.Reifier.reifyTypeArguments(Reifier.java:68)
    at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:138)
    at sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49)
    at sun.reflect.generics.repository.ClassRepository.getSuperclass(ClassRepository.java:88)
    at java.lang.Class.getGenericSuperclass(Class.java:709)
    at com.qubole.rubix.core.CachingFileSystem.getTypeParameterClass(CachingFileSystem.java:75)
    at com.qubole.rubix.core.CachingFileSystem.<init>(CachingFileSystem.java:83)
    at com.qubole.rubix.presto.CachingPrestoS3FileSystem.<init>(CachingPrestoS3FileSystem.java:31)
    at com.qubole.rubix.presto.TestCachingPrestoS3FileSystem.testBasic(TestCachingPrestoS3FileSystem.java:31)
coveralls commented 6 years ago

Pull Request Test Coverage Report for Build 84


Files with Coverage Reduction New Missed Lines %
rubix-bookkeeper/src/main/java/com/qubole/rubix/bookkeeper/FileDownloader.java 1 90.91%
rubix-spi/src/main/java/com/qubole/rubix/spi/DataTransferClientHelper.java 1 0.0%
rubix-core/src/main/java/com/qubole/rubix/core/RemoteFetchRequestChain.java 1 0.0%
rubix-bookkeeper/src/main/java/com/qubole/rubix/bookkeeper/LocalDataTransferServer.java 2 0.0%
rubix-core/src/main/java/com/qubole/rubix/core/CachedReadRequestChain.java 2 84.0%
rubix-core/src/main/java/com/qubole/rubix/core/DirectReadRequestChain.java 2 0.0%
rubix-bookkeeper/src/main/java/com/qubole/rubix/bookkeeper/BookKeeperServer.java 2 0.0%
rubix-bookkeeper/src/main/java/com/qubole/rubix/bookkeeper/utils/ShellExec.java 2 0.0%
rubix-core/src/main/java/com/qubole/rubix/core/CachingInputStream.java 3 0.0%
rubix-core/src/main/java/com/qubole/rubix/core/FileDownloadRequestChain.java 3 0.0%
<!-- Total: 47 -->
Totals Coverage Status
Change from base Build 81: 0.1%
Covered Lines: 548
Relevant Lines: 2203

💛 - Coveralls
jordanw-bq commented 6 years ago

Confirmed that the build passes when built using JDK 8

vrajat commented 6 years ago

I'll let others comment on the change to Java8

shubhamtagra commented 6 years ago

can we test hive with java7 and presto with java8?