SX-24: Support of AWS roles

qubole / streamx

kafka-connect-s3 : Ingest data from Kafka to Object Stores(s3)

Apache License 2.0

96 stars 54 forks source link

SX-24: Support of AWS roles #43

Closed ashishsachdeva closed 7 years ago

ashishsachdeva commented 7 years ago

I have compiled this with qubole hadoop. Open source hadoop has issues with aws roles.

@PraveenSeluka

PraveenSeluka commented 7 years ago

Ashish, you need to make the profile that compiles with qubole-hadoop right. By default, it will use OSS. But if you use the profile, it overrides it. This also means, we will have two streamx docker images, one with OSS hadoop and the other with qubole hadoop (we can start tagging docker images for this)