ABRiS #343 Add support for Apache Spark 4.0.0-SNAPSHOT with fewer exceptions

[ni-mi]

Hi,

This pull request should close issue 343: It seems like the snapshot of Spark 4.0.0 changed (again) the signature of AvroDeserializer's c'tor. I added support for the new c'tor, while changing the code so it won't fail and recover - instead it just looks for a c'tor that is supported and use it. If it can't find a c'tor - it throws NoSuchMethodException - but with a message that describes what c'tors are supported, so it will be easier to support more c'tors in the future.

[ni-mi]

@cerveada / @kevinwallimann I'd appreciate your review on this.

Thanks!

[SlevinBE]

I was experiencing the same issue as mentioned in https://github.com/AbsaOSS/ABRiS/issues/343#issuecomment-2012653699 I've tested this PR with Spark 3.5 on Databricks Runtime 14.3, which contains that new AvroDeserializer, and it solves that issue. Also, great work on the code improvements and the better error reporting 👍

[SlevinBE]

Since this also impacts those using Spark 3.5 when running on Databricks, it would be great to have a 6.4.1 release for this 🙏

[ni-mi]

Thanks @SlevinBE and @cerveada 🙏 Can you merge? AFAIK the CI check JaCoCo code-coverage failure seems to be unrelated - "HttpError: Resource not accessible by integration"

[cerveada]

AFAIK the CI check JaCoCo code-coverage failure seems to be unrelated - "HttpError: Resource not accessible by integration"

yeah, I am not worried about this.

Let's wait for @kevinwallimann

[ni-mi]

@kevinwallimann Can you please merge it - so we might be able to release a 6.4.1 release for Databricks?

Thanks!

[kevinwallimann]

@kevinwallimann Can you please merge it - so we might be able to release a 6.4.1 release for Databricks?

Thanks!

Hi @ni-mi I've merged this PR and have just released a new version 6.4.1 to Maven Central. Thank you very much for your contribution!