search

qubole / streamx

kafka-connect-s3 : Ingest data from Kafka to Object Stores(s3)
Apache License 2.0
96 stars 54 forks source link

Tmp File Double slash #16

Closed ariepratama closed 8 years ago

ariepratama commented 8 years ago

Hi, I got my tmp file looks wrong, it contains double slash in the path like: "s3://airy-poc-ingestion/topics//+tmp/avrotest/partition=0/ad91f606-8ebd-4aab-8598-335d0c566e31_tmp.avro" "s3://airy-poc-ingestion/topics//+tmp/avrotest/partition=0/ad91f606-8ebd-4aab-8598-335d0c566e31_tmp.avro"

commited files looks just fine: "s3://airy-poc-ingestion/topics/avrotest/partition=0/avrotest+0+0000000000+0000000002.avro"

PraveenSeluka commented 8 years ago

Hi @ariepratama

Yes, this is due to an extra slash somewhere while creating the tmp file path. Will fix it. But it should not be causing any issue actually. Let me know if its causing issues. Thanks

ariepratama commented 8 years ago

Hi @PraveenSeluka, actually in my case this will cause issues, because instead of writing to s3://bucket/topics, it will write to s3://bucket//topics. This will make a folder with no name if I see within the s3 console. Is that not happening on your side?

PraveenSeluka commented 8 years ago

Hi @ariepratama Sorry for the late response, I dont see folder with no-name getting created. That said, I will fix this issue shortly

PraveenSeluka commented 8 years ago

This is fixed.