search

qubole / streamx

kafka-connect-s3 : Ingest data from Kafka to Object Stores(s3)
Apache License 2.0
96 stars 54 forks source link

Tests failing? #44

Open thomaska opened 7 years ago

thomaska commented 7 years ago

I am trying to run the tests locally. I just cloned the project, ran mvn test and I get 35 errors. All of them have to do with Unknown configuration 's3.url'. Moreover the stacktrace for one of those errors:

testAlterSchema(io.confluent.connect.hdfs.avro.AvroHiveUtilTest)  Time elapsed: 101.004 sec  <<< ERROR!
org.apache.kafka.common.config.ConfigException: Unknown configuration 's3.url'
    at org.apache.kafka.common.config.AbstractConfig.get(AbstractConfig.java:74)
    at org.apache.kafka.common.config.AbstractConfig.getString(AbstractConfig.java:109)
    at io.confluent.connect.hdfs.hive.HiveUtil.<init>(HiveUtil.java:33)
    at io.confluent.connect.hdfs.avro.AvroHiveUtil.<init>(AvroHiveUtil.java:44)
    at io.confluent.connect.hdfs.avro.AvroHiveUtilTest.setUp(AvroHiveUtilTest.java:48)

I have tried adding the s3.url in quickstart-hdfs.properties with no luck. Is this a known issue maybe? Am I missing something obvious?

UPDATE: It seems that the tests don't pick up the files under config/ folder and they fail because the specific config has no default. Any ideas?

Any help would be greatly appreciated!

jessecollier commented 7 years ago

I came across this too... looks like this very old commit is the trouble maker: https://github.com/qubole/streamx/commit/fa79dbe3f8b8c8ae390f3316a01c7dee1615369e#diff-c57a69bf802558795c71d33cbeb0a740R33

I suppose this project is rather "dead" and not much consideration for passing tests ...

PraveenSeluka commented 7 years ago

@thomaska yes, the tests need to be fixed. I will get it fixed this week. Other than that, StreamX works fine. Can you skipTests now and follow the README to try out in distributed mode.

thomaska commented 7 years ago

@PraveenSeluka thanks! Yes this is what I am already doing, but I would like to add some functionality along with some tests and I wasn't able to ran all tests to see if I broke anything. Thanks again!