Currently we're using the synchronous IO from the Iceberg utils to read Parquet file footers for file metrics. This is very slow when processing many files (e.g. in redrive).
Use Netty Async NIO with S3 to read file footers
Tested to handle 1000 files in ~15 seconds in Lambda (not optimal but OK), happy case of few files as usual (~1s).
(Includes minor change to make sure to drop view on Iceberg table delete)
Signed-off-by: 🐼 Samrose Ahmed 🐼 samroseahmed@gmail.com