search

bkvarda / kudusparklyr

A Kudu extension for Sparklyr
11 stars 5 forks source link

Kudu 1.6.0 with Spark 2.2.0 #7

Closed nachti closed 6 years ago

nachti commented 6 years ago

Hi Brandon! Please add the respective jars for Kudu 1.6.0 on Spark 2.2.0. My admin decided to upgrade Kudu ... Thanks, Gerhard

bkvarda commented 6 years ago

Hi Gerhard,

I'm no longer actively maintaining this repo, so in the future you may want to fork and commit the JAR yourself or feel free to submit a pull request with the change.

For your convenience I've done it for you this time, see https://github.com/bkvarda/kudusparklyr/commit/6f2f56946070b4c85f779a1c3b19b1454ad1af1a

I have not tested it though and will not be able to provide support for it going forward. HTH.

nachti commented 6 years ago

Thank you very much Brandon! It's working for me. Where can I get the required jars in the future to put it in the repo and where is options(kudu.version = "1.6.0") used further on in the code? Cheers, Gerhard

bkvarda commented 6 years ago

Hi Gerhard,

The JARs I use can be found here (or you can build form source): https://mvnrepository.com/artifact/org.apache.kudu/kudu-spark2

options(kudu.version = "1.6.0") helps to choose the proper JAR, and that happens here: https://github.com/bkvarda/kudusparklyr/blob/master/R/dependencies.R#L8

The naming convention for the JARs is important for choosing the correct one.

nachti commented 6 years ago

Thanks Brandon!

I see. That should be possible :-) Looking at the code of dependencies.R I found the naming conventions. Did you rename the jar from the maven repo manually?

~g

bkvarda commented 6 years ago

@nachti Yup that's what I do :) Download and rename (manually)

nachti commented 6 years ago

Ok