Open clairemcginty opened 1 month ago
com.google.cloud.bigdataoss:{gcsio, gcs-connector, util, util-hadoop} libraries have a new 3.x release that supports vectored IO APIs: https://github.com/GoogleCloudDataproc/hadoop-connectors/releases
However, there are some significant breaking changes that will impact Beam's GcsUtil class. Plus, 3.x drops Hadoop 2 support.
Priority: 2 (default / most normal work should be filed as P2)
gcs-connector 3.x drops Java 8 support, so we're blocked on Beam dropping Java 8 as well.
What needs to happen?
com.google.cloud.bigdataoss:{gcsio, gcs-connector, util, util-hadoop} libraries have a new 3.x release that supports vectored IO APIs: https://github.com/GoogleCloudDataproc/hadoop-connectors/releases
However, there are some significant breaking changes that will impact Beam's GcsUtil class. Plus, 3.x drops Hadoop 2 support.
Issue Priority
Priority: 2 (default / most normal work should be filed as P2)
Issue Components