search

springml / spark-sftp

Spark connector for SFTP
Apache License 2.0
100 stars 98 forks source link

Issues using spark-sftp as a Maven dependency #1

Closed yinrayhu closed 8 years ago

yinrayhu commented 8 years ago

There seems to be a bug in build.sbt.

When I add the following dependencies in pom.xml, 

    <dependency>
      <groupId>com.springml</groupId>
      <artifactId>spark-sftp_2.10</artifactId>
      <version>1.0.0</version>
    </dependency>

I get the following WARNING, which ultimately results in ERROR.

[WARNING] The POM for databricks:spark-avro:jar:2.0.1-s_2.10 is missing, no dependency information available
[WARNING] The POM for databricks:spark-csv:jar:1.3.0-s_2.10 is missing, no dependency information available

At line 26 and 27 of build.sbt, 

spDependencies += "databricks/spark-avro:2.0.1-s_2.10"
spDependencies += "databricks/spark-csv:1.3.0-s_2.10"

the two "-s" should be removed.

I already forked the repo and will fix it.

springml commented 8 years ago

Please feel free to submit a pull request if you've fixed it

antonkw commented 8 years ago

Try to fix it. sbt package work good, but I can`t check behavior of checking out from Maven.

tonyfraser commented 7 years ago

Seeing the same in my spark 2 intellij envrionment. Would really love to use this package!

error sbt.ResolveException: unresolved dependency: databricks#spark-avro;2.0.1-s_2.10: not found [error] unresolved dependency: databricks#spark-csv;1.3.0-s_2.10: not found