Closed kevinwallimann closed 7 months ago
Closes #352
Support for the “enableStableIdentifiersForUnionType” option introduced in https://github.com/apache/spark/pull/41263/ was not added. Instead, the option is set to false, which is the default value as documented in https://spark.apache.org/docs/latest/sql-data-sources-avro.html#data-source-option. Users can still use the feature by providing their own SchemaConverter implementation, as explained in https://github.com/absaoss/abris?tab=readme-ov-file#custom-data-conversions If there is demand to directly support this feature through Abris, it can be implemented, but it would require another reflection-based differentiation among Spark versions that support it or not.
Closes #352
Support for the “enableStableIdentifiersForUnionType” option introduced in https://github.com/apache/spark/pull/41263/ was not added. Instead, the option is set to false, which is the default value as documented in https://spark.apache.org/docs/latest/sql-data-sources-avro.html#data-source-option. Users can still use the feature by providing their own SchemaConverter implementation, as explained in https://github.com/absaoss/abris?tab=readme-ov-file#custom-data-conversions If there is demand to directly support this feature through Abris, it can be implemented, but it would require another reflection-based differentiation among Spark versions that support it or not.