search

hortonworks-spark / spark-llap

Apache License 2.0
101 stars 68 forks source link

making hive warehouse connector compatible with spark 2.4 #282

Open carloslima opened 4 years ago

carloslima commented 4 years ago

What changes were proposed in this pull request?

Currently method like executeQuery does not work when running this library with spark 2.4.0 because in spark 2.4.0 some classes have been removed and some classes have been renamed

DataReaderFactory have been renamed to InputPartition 2)DataReader have been renamed to InputPartitionReader 3)SupportsWriteInternalRow have been removed

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

carloslima commented 4 years ago

I did not write the patch, I'm just creating the PR based on #272

lfrancke commented 2 years ago

I know this is old but: @carloslima did you test this and did this work for you with Spark 2.4?

carloslima commented 2 years ago

@lfrancke old, indeed :sweat_smile:

We opted to go a different route so I have no feedback to offer :-)

Please, note that I didn't write this patch. This is all @abh1sh2k work, the only thing I did was open this MR based on their fork that was mentioned in https://github.com/hortonworks-spark/spark-llap/issues/272.

Assuming @abh1sh2k still uses this patch, they may have something to share :-)

lfrancke commented 2 years ago

Thanks anyway! I assumed as much :) The reason I was asking is because I tried it and while it compiles it fails with various errors at runtime.