custom-dynclass-loader - Githubissues

databricks / iceberg-kafka-connect

Apache License 2.0

220 stars 49 forks source link

Closed tabmatfournier closed 8 months ago

tabmatfournier commented 8 months ago

we produce a -hive and a not hive version of the connector. Due to how we strip the packages, some Kafka Connect clusters can throw a NoClassDefFound error when it cannot find hadoop/hive in the non-hive version when trying to detect hive/hadoop (e.g. when using a rest catalog). This is fine, however the reflection code we use only converts ClassPathNotFound to nulls. Wrap the calls to DynClasses to catch the NoClassDefFound due to the jar stripping.

tabmatfournier commented 8 months ago

Tested on confluent cloud:

0.6.13 no hive version published on confluent hub -> throws NoClassDefFound during loading of catalog, even though we don't use hive/hadoop

0.6.13 hive version (manually uploaded) -> works

0.6.13 this version w/ the alternate DynClass manually uploaded, no hive version -> works