search

avast / hdfs-shell

HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS
Apache License 2.0
150 stars 33 forks source link

version 1.0.1 No such file or directory #3

Closed eadjudeanu closed 7 years ago

eadjudeanu commented 7 years ago

Installed v 1.0.1 under /opt and it's failing. v.1.0.0 works fine

/opt/hdfs-shell-1.0.1 ./bin/hdfs-shell.sh -bash: ./bin/hdfs-shell.sh: /bin/bash^M: bad interpreter: No such file or directory cd bin/ ./hdfs-shell.sh -bash: ./hdfs-shell.sh: /bin/bash^M: bad interpreter: No such file or directory

Vity01 commented 7 years ago

Try to update the .sh script this way: https://stackoverflow.com/questions/2920416/configure-bin-shm-bad-interpreter with the :set fileformat=unix and let me know if it works.

eadjudeanu commented 7 years ago

Better but still failing.

./hdfs-shell.sh Error: Could not find or load main class com.avast.server.hdfsshell.MainApp or with absolute paths: /opt/hdfs-shell/bin/hdfs-shell.sh Error: Could not find or load main class com.avast.server.hdfsshell.MainApp

Vity01 commented 7 years ago

Did you extract the zip file properly? There should be lib directory present. I cannot test it, I am running on Windows right now...

eadjudeanu commented 7 years ago

$[hdfs-shell-1.0.1]$ ls -altR .: total 8 drwxr-xr-x. 2 root root 53 Aug 24 16:41 bin drwxr-xr-x. 9 root root 4096 Aug 24 16:39 .. drwxr-xr-x. 2 root root 4096 Aug 24 16:39 lib drwxr-xr-x. 4 root root 26 Apr 12 08:19 .

./bin: total 8 drwxr-xr-x. 2 root root 53 Aug 24 16:41 . -rwxr-xr-x. 1 root root 141 Aug 24 16:41 hdfs-shell.sh drwxr-xr-x. 4 root root 26 Apr 12 08:19 .. -rw-r--r--. 1 root root 159 Apr 12 08:15 hdfs-shell-daemon.sh

./lib: total 42244 drwxr-xr-x. 2 root root 4096 Aug 24 16:39 . drwxr-xr-x. 4 root root 26 Apr 12 08:19 .. -rw-r--r--. 1 root root 31631 Apr 12 08:19 hdfs-shell-1.0.jar -rw-r--r--. 1 root root 18216 Jan 24 2017 junixsocket-common-2.0.4.jar -rw-r--r--. 1 root root 27805 Jan 24 2017 junixsocket-native-common-2.0.4.jar -rw-r--r--. 1 root root 17052 Jan 24 2017 native-lib-loader-2.0.2.jar -rw-r--r--. 1 root root 231952 Jan 17 2017 gson-2.7.jar -rw-r--r--. 1 root root 220536 Jan 17 2017 xml-apis-1.4.01.jar -rw-r--r--. 1 root root 2290 Jan 16 2017 spring-boot-starter-1.4.3.RELEASE.jar -rw-r--r--. 1 root root 246174 Jan 16 2017 commons-beanutils-1.9.3.jar -rw-r--r--. 1 root root 196768 Jan 16 2017 commons-digester-2.1.jar -rw-r--r--. 1 root root 187150 Jan 16 2017 spring-context-support-4.3.4.RELEASE.jar -rw-r--r--. 1 root root 9988 Jan 16 2017 slf4j-log4j12-1.7.21.jar -rw-r--r--. 1 root root 195025 Jan 13 2017 spring-shell-1.2.0.RELEASE.jar -rw-r--r--. 1 root root 2243036 Jan 10 2017 guava-17.0.jar -rw-r--r--. 1 root root 761236 Nov 9 2016 spring-beans-4.3.4.RELEASE.jar -rw-r--r--. 1 root root 263392 Nov 9 2016 spring-expression-4.3.4.RELEASE.jar -rw-r--r--. 1 root root 380082 Nov 9 2016 spring-aop-4.3.4.RELEASE.jar -rw-r--r--. 1 root root 1135493 Nov 9 2016 spring-context-4.3.4.RELEASE.jar -rw-r--r--. 1 root root 658888 Nov 9 2016 spring-boot-1.4.2.RELEASE.jar -rw-r--r--. 1 root root 968499 Nov 9 2016 spring-boot-autoconfigure-1.4.2.RELEASE.jar -rw-r--r--. 1 root root 2311 Nov 9 2016 spring-boot-starter-logging-1.4.2.RELEASE.jar -rw-r--r--. 1 root root 1112602 Nov 9 2016 spring-core-4.3.4.RELEASE.jar -rw-r--r--. 1 root root 995968 Sep 23 2016 snappy-java-1.0.4.1.jar -rw-r--r--. 1 root root 273599 Sep 5 2016 snakeyaml-1.17.jar -rw-r--r--. 1 root root 327373 Sep 5 2016 httpcore-4.4.5.jar -rw-r--r--. 1 root root 213854 Jul 29 2016 jline-2.12.jar -rw-r--r--. 1 root root 532497 Jul 29 2016 protobuf-java-2.5.0.jar -rw-r--r--. 1 root root 16431 Jun 2 2016 jcl-over-slf4j-1.7.21.jar -rw-r--r--. 1 root root 4597 Jun 2 2016 jul-to-slf4j-1.7.21.jar -rw-r--r--. 1 root root 736658 May 27 2016 httpclient-4.5.2.jar -rw-r--r--. 1 root root 41071 May 23 2016 slf4j-api-1.7.21.jar -rw-r--r--. 1 root root 588337 May 13 2016 commons-collections-3.2.2.jar -rw-r--r--. 1 root root 29555 Jan 21 2016 paranamer-2.3.jar -rw-r--r--. 1 root root 284184 Aug 24 2015 commons-codec-1.10.jar -rw-r--r--. 1 root root 185245 Aug 6 2015 curator-framework-2.6.0.jar -rw-r--r--. 1 root root 1208356 Aug 6 2015 netty-3.7.0.Final.jar -rw-r--r--. 1 root root 44925 Aug 6 2015 apacheds-i18n-2.0.0-M15.jar -rw-r--r--. 1 root root 16560 Aug 6 2015 api-asn1-api-1.0.0-M20.jar -rw-r--r--. 1 root root 79912 Aug 6 2015 api-util-1.0.0-M20.jar -rw-r--r--. 1 root root 68866 Aug 6 2015 curator-client-2.6.0.jar -rw-r--r--. 1 root root 248171 Aug 6 2015 curator-recipes-2.6.0.jar -rw-r--r--. 1 root root 18336 Aug 6 2015 jackson-jaxrs-1.9.13.jar -rw-r--r--. 1 root root 27084 Aug 6 2015 jackson-xc-1.9.13.jar -rw-r--r--. 1 root root 1045744 Aug 6 2015 leveldbjni-all-1.8.jar -rw-r--r--. 1 root root 105112 Aug 6 2015 servlet-api-2.5.jar -rw-r--r--. 1 root root 23346 Aug 6 2015 stax-api-1.0-2.jar -rw-r--r--. 1 root root 792964 Aug 6 2015 zookeeper-3.4.6.jar -rw-r--r--. 1 root root 691479 Aug 6 2015 apacheds-kerberos-codec-2.0.0-M15.jar -rw-r--r--. 1 root root 284220 Aug 6 2015 commons-lang-2.6.jar -rw-r--r--. 1 root root 1599627 Aug 6 2015 commons-math3-3.1.1.jar -rw-r--r--. 1 root root 17035 Aug 6 2015 hadoop-annotations-2.6.0.jar -rw-r--r--. 1 root root 67167 Aug 6 2015 hadoop-auth-2.6.0.jar -rw-r--r--. 1 root root 2556 Aug 6 2015 hadoop-client-2.6.0.jar -rw-r--r--. 1 root root 3360985 Aug 6 2015 hadoop-common-2.6.0.jar -rw-r--r--. 1 root root 7822670 Aug 6 2015 hadoop-hdfs-2.6.0.jar -rw-r--r--. 1 root root 504309 Aug 6 2015 hadoop-mapreduce-client-app-2.6.0.jar -rw-r--r--. 1 root root 664918 Aug 6 2015 hadoop-mapreduce-client-common-2.6.0.jar -rw-r--r--. 1 root root 1509399 Aug 6 2015 hadoop-mapreduce-client-core-2.6.0.jar -rw-r--r--. 1 root root 37225 Aug 6 2015 hadoop-mapreduce-client-jobclient-2.6.0.jar -rw-r--r--. 1 root root 44362 Aug 6 2015 hadoop-mapreduce-client-shuffle-2.6.0.jar -rw-r--r--. 1 root root 1870176 Aug 6 2015 hadoop-yarn-api-2.6.0.jar -rw-r--r--. 1 root root 127986 Aug 6 2015 hadoop-yarn-client-2.6.0.jar -rw-r--r--. 1 root root 1602059 Aug 6 2015 hadoop-yarn-common-2.6.0.jar -rw-r--r--. 1 root root 289619 Aug 6 2015 hadoop-yarn-server-common-2.6.0.jar -rw-r--r--. 1 root root 31212 Aug 6 2015 htrace-core-3.0.4.jar -rw-r--r--. 1 root root 1229125 Aug 6 2015 xercesImpl-2.9.1.jar -rw-r--r--. 1 root root 62983 Jul 29 2015 activation-1.1.jar -rw-r--r--. 1 root root 273370 Jul 29 2015 commons-net-3.1.jar -rw-r--r--. 1 root root 105134 Jul 29 2015 jaxb-api-2.2.2.jar -rw-r--r--. 1 root root 130458 Jul 29 2015 jersey-client-1.9.jar -rw-r--r--. 1 root root 458739 Jul 29 2015 jersey-core-1.9.jar -rw-r--r--. 1 root root 303139 Jul 29 2015 avro-1.7.4.jar -rw-r--r--. 1 root root 206035 Jul 29 2015 commons-beanutils-core-1.8.0.jar -rw-r--r--. 1 root root 41123 Jul 29 2015 commons-cli-1.2.jar -rw-r--r--. 1 root root 241367 Jul 29 2015 commons-compress-1.4.1.jar -rw-r--r--. 1 root root 298829 Jul 29 2015 commons-configuration-1.6.jar -rw-r--r--. 1 root root 177131 Jul 29 2015 jetty-util-6.1.26.jar -rw-r--r--. 1 root root 15010 Jul 29 2015 xmlenc-0.52.jar -rw-r--r--. 1 root root 94672 Jul 29 2015 xz-1.0.jar -rw-r--r--. 1 root root 185140 Jul 29 2015 commons-io-2.4.jar -rw-r--r--. 1 root root 305001 Jul 29 2015 commons-httpclient-3.1.jar -rw-r--r--. 1 root root 33015 Jul 29 2015 jsr305-1.3.9.jar -rw-r--r--. 1 root root 489884 Jul 29 2015 log4j-1.2.17.jar -rw-r--r--. 1 root root 232248 May 5 2015 jackson-core-asl-1.9.13.jar -rw-r--r--. 1 root root 780664 May 5 2015 jackson-mapper-asl-1.9.13.jar -rw-r--r--. 1 root root 61829 Mar 30 2015 commons-logging-1.2.jar

I'll try on another machine but i did use the zip file provided in releases.

Vity01 commented 7 years ago

Just use the hdfs-shell.sh I uploaded to this comment. I will fix the .sh later.

   #!/bin/bash

java -Xms200m -Xmx400m -cp ./lib/*:/etc/hadoop/conf com.avast.server.hdfsshell.MainApp "$@"

`

eadjudeanu commented 7 years ago

the above works but you need to be outside of the bin dir. Also it doesn't work with absolute paths. I think we need a better solution if possible.

Example: $/opt/hdfs-shell/bin/hdfs-shell.sh Error: Could not find or load main class com.avast.server.hdfsshell.MainApp

eadjudeanu commented 7 years ago

This works:

!/bin/bash

MY_DIR=dirname $0 java -Xms200m -Xmx400m -cp $MY_DIR/../lib/*:/etc/hadoop/conf com.avast.server.hdfsshell.MainApp "$@"

Please test it and create a new release. Otherwise the app is brilliant. Thanks for it.

Vity01 commented 7 years ago

I just released version 1.0.2 which hopefully will resolve the problems. Thanks for reporting. Enjoy it.