search

univalence / zio-spark

A functional wrapper around Spark to make it works with ZIO
https://univalence.github.io/zio-spark/
Apache License 2.0
42 stars 9 forks source link

java.io.FileNotFoundException: hadoop-client-api-3.3.1.jar #324

Open ahoy-jon opened 1 year ago

ahoy-jon commented 1 year ago

I keep having those errors (can be reproduced on the template g8), is there a workaround?

java.io.FileNotFoundException: /Users/jon/Documents/GitHub/testZioSparkM1/zio-spark-project/zio-spark-project/target/bg-jobs/sbt_e1512462/target/c0a2de93/81ecd14d/hadoop-client-api-3.3.1.jar (No such file or directory)
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:233)
        at java.util.zip.ZipFile.<init>(ZipFile.java:162)
        at java.util.jar.JarFile.<init>(JarFile.java:173)
        at java.util.jar.JarFile.<init>(JarFile.java:110)
        at sun.net.www.protocol.jar.URLJarFile.<init>(URLJarFile.java:93)
        at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:69)
        at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:99)
        at sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:122)
        at sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:152)
        at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2986)
        at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:3082)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3040)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:3013)
        at org.apache.hadoop.conf.Configuration.loadProps(Configuration.java:2893)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2875)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1225)
        at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1842)
        at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1819)
        at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
        at org.apache.hadoop.util.ShutdownHookManager.shutdownExecutor(ShutdownHookManager.java:145)
        at org.apache.hadoop.util.ShutdownHookManager.access$300(ShutdownHookManager.java:65)
        at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:102)
Exception in thread "Thread-16" java.lang.RuntimeException: java.io.FileNotFoundException: /Users/jon/Documents/GitHub/testZioSparkM1/zio-spark-project/zio-spark-project/target/bg-jobs/sbt_e1512462/target/c0a2de93/81ecd14d/hadoop-client-api-3.3.1.jar (No such file or directory)
ahoy-jon commented 1 year ago

okay, some dude found something 6 months ago on the internet : https://github.com/sbt/sbt/issues/6888#issuecomment-1176377545 need to check and update the templates.