search

oracle / oci-hdfs-connector

HDFS Connector for Oracle Cloud Infrastructure
https://cloud.oracle.com/cloud-infrastructure
Other
27 stars 26 forks source link

Support for Apache Flink #27

Closed rohitagl91 closed 4 years ago

rohitagl91 commented 4 years ago

Wanted to know if there is a similar library for integrating Apache flink with OCI Object Storage?

tsammut commented 4 years ago

Hi @rohitagl91.

There is not, sorry. I am not familiar with Flink, but it appears it supports S3.

https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/s3.html

It is likely, this will work with OCI's Amazon S3 Compatible API.

https://docs.cloud.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm

Make sure Flink is configured for path-style access. Also, since, at least by looking at the Flink docs, Flink does not support specifying a "region name" like us-phoenix-1, Flink+S3 will only work your OCI tenancy's home region.

hope this helps!

mricken commented 4 years ago

Thanks for your help, @tsammut . @rohitagl91 , since this is not directly related to the HDFS connector, I will close this issue. Please feel free to re-open to discuss the HDFS connector.