Is that java/Scala project use jfs client on hadoop need hadoop environment?

juicedata / juicefs

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

https://juicefs.com

Apache License 2.0

10.95k stars 967 forks source link

Is that java/Scala project use jfs client on hadoop need hadoop environment? #5307

Open ming-bug opened 3 days ago

ming-bug commented 3 days ago

When I deploy a java project to visit jfs by java client. I found that if the java demo config with the docs as bellow not enough. I have to add Hadoop-related configurations, otherwise it will report 'java.io.IOException: JuiceFS initialized failed for jfs://myjfs'. So I want to know which hadoop-related configurations are needed. Actually I might deploy my java project outside of my hadoop environment.

tangyoupeng commented 1 day ago

hadoop common is needed because juicefs did not inclue this for version compatiable reason.

ming-bug commented 1 day ago

when I config hbase use jfs, the hbase master reboot failed and some log as bellow. hacluster is my hdfs HA nameservices. Can you help me to resolve it, thanks object storage: new HDFS client [hacluster]: no available namenodes: dial tcp: address hacluster: missing port in address [main.go:503] | io.juicefs.JuiceFileSystemImpl$LogCallBackImpl.call(JuiceFileSystemImpl.java:230)

tangyoupeng commented 10 hours ago

did you use hdfs as the underlying storage？If so, you need configure HADOOP_CONF_DIR env