The only thing that prevented us from employing the relevant DataFusion functions entirely was our custom file_extension setting for HTTP files/tables (to allow disabling filtering by an extension). Previously we copied a lot of code just to make that small change, but now it works as desired in DF, thanks to https://github.com/apache/arrow-datafusion/pull/6274.
Also add a test targeting minio as the host of the HTTP file.
The only thing that prevented us from employing the relevant DataFusion functions entirely was our custom
file_extensionsetting for HTTP files/tables (to allow disabling filtering by an extension). Previously we copied a lot of code just to make that small change, but now it works as desired in DF, thanks to https://github.com/apache/arrow-datafusion/pull/6274.Also add a test targeting minio as the host of the HTTP file.