Closed gerrymiller closed 7 years ago
If you have any other suggestions I can try those when I get back online.
You can try with an older version to see if this is specific to 2.8.1.
git -C $(brew --repo homebrew/core) checkout fe4ca4c -- Formula/hadoop.rb
HOMEBREW_NO_AUTO_UPDATE=1 brew install hadoop
That should try 2.8.0.
Looks like the reference to 2.8.0 isn't correct:
| => git -C $(brew --repo homebrew/core) checkout fe4ca4c -- Formula/hadoop.rb
fatal: invalid reference: fe4ca4c
Any idea what the actual commit ID for 2.8.0 is? I didn't see any release labels or tags, so I'm not sure how that's tracked with homebrew
.
Thanks again.
brew tap --full homebrew/core
should fix that.
Bingo:
| => brew tap --full homebrew/core
==> Unshallowing homebrew/core
remote: Counting objects: 232792, done.
remote: Compressing objects: 100% (111977/111977), done.
remote: Total 232792 (delta 119162), reused 231572 (delta 118031), pack-reused 0
Receiving objects: 100% (232792/232792), 49.51 MiB | 1.65 MiB/s, done.
Resolving deltas: 100% (119162/119162), completed with 3174 local objects.
______________________________
| ~ @ Gerry's MacBook (gerry)
| => git -C $(brew --repo homebrew/core) checkout fe4ca4c -- Formula/hadoop.rb
______________________________
| ~ @ Gerry's MacBook (gerry)
| => HOMEBREW_NO_AUTO_UPDATE=1 brew install hadoop
==> Downloading https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz
==> Best Mirror http://www-eu.apache.org/dist/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz
######################################################################## 100.0%
==> Caveats
In Hadoop's config file:
/usr/local/opt/hadoop/libexec/etc/hadoop/hadoop-env.sh,
/usr/local/opt/hadoop/libexec/etc/hadoop/mapred-env.sh and
/usr/local/opt/hadoop/libexec/etc/hadoop/yarn-env.sh
$JAVA_HOME has been set to be the output of:
/usr/libexec/java_home
==> Summary
🍺 /usr/local/Cellar/hadoop/2.8.0: 25,169 files, 2.1GB, built in 6 minutes 49 seconds
Any thoughts on what might have broken this in 2.8.1?
Not yet :)
Can you try this?
mkdir /tmp/hadoop-tmp
cp ~/Library/Caches/Homebrew/hadoop-2.8.1.tar.gz /tmp/hadoop-tmp/
cd /tmp/hadoop-tmp
tar xf hadoop-2.8.1.tar.gz
cd hadoop-2.8.1
Then run brew irb
iMac-TMP:hadoop-2.8.1 joe$ brew irb
==> Interactive Homebrew Shell
Example commands available with: brew irb --examples
irb(main):001:0> Pathname.pwd.to_enum(:find).select(&:file?).map(&:mtime).max
=> 2017-06-01 23:49:58 -0700
irb(main):002:0>
| /tmp/hadoop-tmp @ Gerry's MacBook (gerry)
| => cd hadoop-2.8.1
total 120
-rw-r--r-- 1 gerry wheel 97K Jun 2 08:24 LICENSE.txt
-rw-r--r-- 1 gerry wheel 16K Jun 2 08:24 NOTICE.txt
-rw-r--r-- 1 gerry wheel 1.3K Jun 2 08:24 README.txt
drwxr-xr-x 13 gerry wheel 442B Jun 2 08:24 bin/
drwxr-xr-x 3 gerry wheel 102B Jun 2 08:24 etc/
drwxr-xr-x 7 gerry wheel 238B Jun 2 08:24 include/
drwxr-xr-x 3 gerry wheel 102B Jun 2 08:24 lib/
drwxr-xr-x 12 gerry wheel 408B Jun 2 08:24 libexec/
drwxr-xr-x 30 gerry wheel 1.0K Jun 2 08:24 sbin/
drwxr-xr-x 4 gerry wheel 136B Jun 2 08:24 share/
______________________________
| /tmp/hadoop-tmp/hadoop-2.8.1 @ Gerry's MacBook (gerry)
| => brew irb
==> Interactive Homebrew Shell
Example commands available with: brew irb --examples
irb(main):001:0> Pathname.pwd.to_enum(:find).select(&:file?).map(&:mtime).max
=> 2017-06-02 08:49:58 +0200
irb(main):002:0>
OK. And what happens if you
cd $(brew --repo homebrew/core)
git reset --hard origin/master
brew upgrade hadoop
Back to failure:
| => cd $(brew --repo homebrew/core)
total 32
drwxr-xr-x 15 gerry wheel 510B Sep 6 11:10 .git/
drwxr-xr-x 5 gerry wheel 170B Aug 28 06:11 .github/
drwxr-xr-x 228 gerry wheel 7.6K Sep 6 10:39 Aliases/
-rw-r--r-- 1 gerry wheel 3.1K Apr 27 17:14 CODEOFCONDUCT.md
-rw-r--r-- 1 gerry wheel 1.6K Aug 8 16:04 CONTRIBUTING.md
drwxr-xr-x 4343 gerry wheel 144K Sep 6 11:10 Formula/
-rw-r--r-- 1 gerry wheel 2.7K Apr 27 17:14 Jenkinsfile
-rw-r--r-- 1 gerry wheel 1.3K Apr 27 17:14 LICENSE.txt
-rw-r--r-- 1 gerry wheel 1.0K Aug 8 16:04 README.md
-rw-r--r-- 1 gerry wheel 4.8K Sep 4 08:27 formula_renames.json
-rw-r--r-- 1 gerry wheel 1.7K Aug 21 16:04 tap_migrations.json
______________________________
| /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core @ Gerry's MacBook (gerry)
| => git reset --hard origin/master
HEAD is now at d914b456d5 cheat: update 2.2.1 bottle.
______________________________
| /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core @ Gerry's MacBook (gerry)
| => brew upgrade hadoop
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).
==> Updated Formulae
cmark-gfm crash libxml2 ✔ opendetex telegraf trafficserver valabind
==> Upgrading 1 outdated package, with result:
hadoop 2.8.1
==> Upgrading hadoop
==> Downloading https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz
Already downloaded: /Users/gerry/Library/Caches/Homebrew/hadoop-2.8.1.tar.gz
Error: cannot load such file -- find
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/lib/ruby/2.0.0/pathname.rb:542:in `find'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:51:in `each'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:51:in `select'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:51:in `source_modified_time'
/usr/local/Homebrew/Library/Homebrew/resource.rb:95:in `block in unpack'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:14:in `block in mktemp'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in `block in run'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in `chdir'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in `run'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:13:in `mktemp'
/usr/local/Homebrew/Library/Homebrew/resource.rb:93:in `unpack'
/usr/local/Homebrew/Library/Homebrew/resource.rb:85:in `stage'
/usr/local/Homebrew/Library/Homebrew/formula.rb:1887:in `stage'
/usr/local/Homebrew/Library/Homebrew/formula.rb:1088:in `brew'
/usr/local/Homebrew/Library/Homebrew/build.rb:114:in `block in install'
/usr/local/Homebrew/Library/Homebrew/utils.rb:554:in `with_env'
/usr/local/Homebrew/Library/Homebrew/build.rb:111:in `install'
/usr/local/Homebrew/Library/Homebrew/build.rb:188:in `<main>'
OK. And what does this do now
HOMEBREW_ENV_FILTERING=1 HOMEBREW_NO_AUTO_UPDATE=1 brew upgrade -dv hadoop
| => HOMEBREW_ENV_FILTERING=1 HOMEBREW_NO_AUTO_UPDATE=1 brew upgrade -dv hadoop
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/hadoop.rb
==> Upgrading 1 outdated package, with result:
hadoop 2.8.1
==> Upgrading hadoop
rm /usr/local/bin/container-executor
rm /usr/local/bin/hadoop
rm /usr/local/bin/hdfs
rm /usr/local/bin/mapred
rm /usr/local/bin/test-container-executor
rm /usr/local/bin/yarn
rm /usr/local/sbin/distribute-exclude.sh
rm /usr/local/sbin/hadoop-daemon.sh
rm /usr/local/sbin/hadoop-daemons.sh
rm /usr/local/sbin/hdfs-config.sh
rm /usr/local/sbin/httpfs.sh
rm /usr/local/sbin/kms.sh
rm /usr/local/sbin/mr-jobhistory-daemon.sh
rm /usr/local/sbin/refresh-namenodes.sh
rm /usr/local/sbin/slaves.sh
rm /usr/local/sbin/start-all.sh
rm /usr/local/sbin/start-balancer.sh
rm /usr/local/sbin/start-dfs.sh
rm /usr/local/sbin/start-secure-dns.sh
rm /usr/local/sbin/start-yarn.sh
rm /usr/local/sbin/stop-all.sh
rm /usr/local/sbin/stop-balancer.sh
rm /usr/local/sbin/stop-dfs.sh
rm /usr/local/sbin/stop-secure-dns.sh
rm /usr/local/sbin/stop-yarn.sh
rm /usr/local/sbin/yarn-daemon.sh
rm /usr/local/sbin/yarn-daemons.sh
/usr/bin/sandbox-exec -f /Users/gerry/mytemp/homebrew20170906-51178-1kk8zh8.sb nice /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/bin/ruby -W0 -I /usr/local/Homebrew/Library/Homebrew -- /usr/local/Homebrew/Library/Homebrew/build.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/hadoop.rb --verbose --debug
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FromPathLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/hadoop.rb
==> Downloading https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz
Already downloaded: /Users/gerry/Library/Caches/Homebrew/hadoop-2.8.1.tar.gz
==> Verifying hadoop-2.8.1.tar.gz checksum
tar xzf /Users/gerry/Library/Caches/Homebrew/hadoop-2.8.1.tar.gz
ln -s ../Cellar/hadoop/2.8.0/bin/container-executor container-executor
ln -s ../Cellar/hadoop/2.8.0/bin/hadoop hadoop
ln -s ../Cellar/hadoop/2.8.0/bin/hdfs hdfs
ln -s ../Cellar/hadoop/2.8.0/bin/mapred mapred
ln -s ../Cellar/hadoop/2.8.0/bin/test-container-executor test-container-executor
ln -s ../Cellar/hadoop/2.8.0/bin/yarn yarn
ln -s ../Cellar/hadoop/2.8.0/sbin/distribute-exclude.sh distribute-exclude.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/hadoop-daemon.sh hadoop-daemon.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/hadoop-daemons.sh hadoop-daemons.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/hdfs-config.sh hdfs-config.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/httpfs.sh httpfs.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/kms.sh kms.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/mr-jobhistory-daemon.sh mr-jobhistory-daemon.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/refresh-namenodes.sh refresh-namenodes.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/slaves.sh slaves.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/start-all.sh start-all.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/start-balancer.sh start-balancer.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/start-dfs.sh start-dfs.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/start-secure-dns.sh start-secure-dns.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/start-yarn.sh start-yarn.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/stop-all.sh stop-all.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/stop-balancer.sh stop-balancer.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/stop-dfs.sh stop-dfs.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/stop-secure-dns.sh stop-secure-dns.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/stop-yarn.sh stop-yarn.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/yarn-daemon.sh yarn-daemon.sh
ln -s ../Cellar/hadoop/2.8.0/sbin/yarn-daemons.sh yarn-daemons.sh
Error: cannot load such file -- find
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/lib/ruby/2.0.0/pathname.rb:542:in `find'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:51:in `each'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:51:in `select'
/usr/local/Homebrew/Library/Homebrew/download_strategy.rb:51:in `source_modified_time'
/usr/local/Homebrew/Library/Homebrew/resource.rb:95:in `block in unpack'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:14:in `block in mktemp'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in `block in run'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in `chdir'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in `run'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:13:in `mktemp'
/usr/local/Homebrew/Library/Homebrew/resource.rb:93:in `unpack'
/usr/local/Homebrew/Library/Homebrew/resource.rb:85:in `stage'
/usr/local/Homebrew/Library/Homebrew/formula.rb:1887:in `stage'
/usr/local/Homebrew/Library/Homebrew/formula.rb:1088:in `brew'
/usr/local/Homebrew/Library/Homebrew/build.rb:114:in `block in install'
/usr/local/Homebrew/Library/Homebrew/utils.rb:554:in `with_env'
/usr/local/Homebrew/Library/Homebrew/build.rb:111:in `install'
/usr/local/Homebrew/Library/Homebrew/build.rb:188:in `<main>'
I'll be AFK for a few hours, and will try any additional suggestions you have this evening European Central Time. Thank you!
What's the output of gem list
?
| => gem list
*** LOCAL GEMS ***
aws-sdk (2.2.27)
aws-sdk-core (2.2.27)
aws-sdk-resources (2.2.27)
bigdecimal (1.2.0)
brewdler (0.4.1)
bundler (1.11.2, 1.3.5)
CFPropertyList (2.2.8)
chunky_png (1.2.8)
commander (4.1.3)
compass (0.12.2)
daemons (1.1.9)
eventmachine (1.0.3)
fssm (0.2.10)
highline (1.6.19)
io-console (0.4.2)
jmespath (1.1.3)
json (1.7.7)
libxml-ruby (2.6.0)
minitest (4.3.2)
nokogiri (1.5.6)
psych (2.0.0)
rack (1.1.0)
rack-try_static (0.1.1)
rake (10.5.0, 0.9.6)
rdoc (4.0.0)
sass (3.2.9)
sqlite3 (1.3.7)
test-unit (2.0.0.0)
thin (1.5.1)
thor (0.19.1)
Can you post the output of env
but be sure to strip out any API token, etc.?
| => env
TERM_PROGRAM=Apple_Terminal
HOMEBREW_TEMP=/Users/gerry/mytemp
SHELL=/bin/bash
TERM=xterm-256color
TMPDIR=/var/folders/cl/rk8c_n65471274j6vh56b7b80000gn/T/
Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.7NaoMsqwW7/Render
TERM_PROGRAM_VERSION=388.1.1
OLDPWD=/Users/gerry
TERM_SESSION_ID=<snip>
SVN_EDITOR=atom
USER=gerry
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.PO3w1cllTL/Listeners
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
LSCOLORS=GxFxCxDxBxegedabagaced
PATH=/usr/local/bin:/usr/local/sbin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/git/bin:/usr/local/MacGPG2/bin:/bin
BLOCKSIZE=1k
SCRIPT_HOME=/usr/local/scripts
PWD=/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core
EDITOR=/usr/local/bin/atom
LANG=en_US.UTF-8
MONO_GAC_PREFIX=/usr/local
XPC_FLAGS=0x0
PS1=______________________________\n| \w @ \h (\u) \n| =>
PS2=| =>
XPC_SERVICE_NAME=0
SHLVL=1
HOME=/Users/gerry
GREP_OPTIONS=-D skip --binary-files=without-match --ignore-case
OpenSSL_HOME=/usr/local/ssl
LOGNAME=gerry
_=/usr/bin/env
Hi @ilovezfs, I'm online for the next few hours if you have any more suggestions.
@gerrymiller does this work?
diff --git a/Formula/hadoop.rb b/Formula/hadoop.rb
index 5d578a06a3..d3e93a2665 100644
--- a/Formula/hadoop.rb
+++ b/Formula/hadoop.rb
@@ -1,7 +1,8 @@
class Hadoop < Formula
desc "Framework for distributed processing of large data sets"
homepage "https://hadoop.apache.org/"
- url "https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz"
+ url "https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz",
+ :using => :nounzip
mirror "https://archive.apache.org/dist/hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz"
sha256 "b5be527578ef2c8565cd435ab4ae71a024cad704f7bf597bd792082703bd77d9"
@@ -10,6 +11,8 @@ class Hadoop < Formula
depends_on :java => "1.7+"
def install
+ system "tar", "xf", "hadoop-2.8.1.tar.gz"
+ cd "hadoop-2.8.1"
rm_f Dir["bin/*.cmd", "sbin/*.cmd", "libexec/*.cmd", "etc/hadoop/*.cmd"]
libexec.install %w[bin sbin libexec share etc]
bin.write_exec_script Dir["#{libexec}/bin/*"]
Nope:
| => brew upgrade hadoop
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 2 taps (caskroom/cask, homebrew/core).
==> Updated Formulae
ace lynis
==> Upgrading 1 outdated package, with result:
hadoop 2.8.1
==> Upgrading hadoop
==> Downloading https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz
Already downloaded: /Users/gerry/Library/Caches/Homebrew/hadoop-2.8.1.tar.gz
==> tar xf hadoop-2.8.1.tar.gz
Error: Interrupted system call - /usr/local/Cellar/hadoop/2.8.1/libexec/etc/hadoop/hadoop-env.sh
What's the output of
git -C $(brew --repo homebrew/core) diff
| => git -C $(brew --repo homebrew/core) diff
diff --git a/Formula/hadoop.rb b/Formula/hadoop.rb
index 5d578a06a3..d3e93a2665 100644
--- a/Formula/hadoop.rb
+++ b/Formula/hadoop.rb
@@ -1,7 +1,8 @@
class Hadoop < Formula
desc "Framework for distributed processing of large data sets"
homepage "https://hadoop.apache.org/"
- url "https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz"
+ url "https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz",
+ :using => :nounzip
mirror "https://archive.apache.org/dist/hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz"
sha256 "b5be527578ef2c8565cd435ab4ae71a024cad704f7bf597bd792082703bd77d9"
@@ -10,6 +11,8 @@ class Hadoop < Formula
depends_on :java => "1.7+"
def install
+ system "tar", "xf", "hadoop-2.8.1.tar.gz"
+ cd "hadoop-2.8.1"
rm_f Dir["bin/*.cmd", "sbin/*.cmd", "libexec/*.cmd", "etc/hadoop/*.cmd"]
libexec.install %w[bin sbin libexec share etc]
bin.write_exec_script Dir["#{libexec}/bin/*"]
OK, try running
brew upgrade -dv hadoop
also you should set HOMEBREW_NO_AUTO_UPDATE=1 while doing this
Which action do you want me to choose?
| => HOMEBREW_NO_AUTO_UPDATE=1 brew upgrade -dv hadoop
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/hadoop.rb
==> Upgrading 1 outdated package, with result:
hadoop 2.8.1
==> Upgrading hadoop
rm /usr/local/bin/container-executor
rm /usr/local/bin/hadoop
rm /usr/local/bin/hdfs
rm /usr/local/bin/mapred
rm /usr/local/bin/test-container-executor
rm /usr/local/bin/yarn
rm /usr/local/sbin/distribute-exclude.sh
rm /usr/local/sbin/hadoop-daemon.sh
rm /usr/local/sbin/hadoop-daemons.sh
rm /usr/local/sbin/hdfs-config.sh
rm /usr/local/sbin/httpfs.sh
rm /usr/local/sbin/kms.sh
rm /usr/local/sbin/mr-jobhistory-daemon.sh
rm /usr/local/sbin/refresh-namenodes.sh
rm /usr/local/sbin/slaves.sh
rm /usr/local/sbin/start-all.sh
rm /usr/local/sbin/start-balancer.sh
rm /usr/local/sbin/start-dfs.sh
rm /usr/local/sbin/start-secure-dns.sh
rm /usr/local/sbin/start-yarn.sh
rm /usr/local/sbin/stop-all.sh
rm /usr/local/sbin/stop-balancer.sh
rm /usr/local/sbin/stop-dfs.sh
rm /usr/local/sbin/stop-secure-dns.sh
rm /usr/local/sbin/stop-yarn.sh
rm /usr/local/sbin/yarn-daemon.sh
rm /usr/local/sbin/yarn-daemons.sh
/usr/bin/sandbox-exec -f /Users/gerry/mytemp/homebrew20170907-8510-vjd2cp.sb nice /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/bin/ruby -W0 -I /usr/local/Homebrew/Library/Homebrew -- /usr/local/Homebrew/Library/Homebrew/build.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/hadoop.rb --verbose --debug
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FromPathLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/hadoop.rb
==> Downloading https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz
Already downloaded: /Users/gerry/Library/Caches/Homebrew/hadoop-2.8.1.tar.gz
==> Verifying hadoop-2.8.1.tar.gz checksum
==> tar xf hadoop-2.8.1.tar.gz
/usr/local/Homebrew/Library/Homebrew/utils/inreplace.rb:23:in `initialize'
Errno::EINTR: Interrupted system call - /usr/local/Cellar/hadoop/2.8.1/libexec/etc/hadoop/hadoop-env.sh
1. raise
2. backtrace
3. shell
Choose an action:
Choose 3 and then ls
Choose an action: 3
When you exit this shell, you will return to the menu.
bash-3.2$ ls
LICENSE.txt NOTICE.txt README.txt include lib
bash-3.2$
ls -al /usr/local/Cellar/hadoop/2.8.1/libexec/etc/hadoop/hadoop-env.sh
bash-3.2$ ls -al /usr/local/Cellar/hadoop/2.8.1/libexec/etc/hadoop/hadoop-env.sh
-rw-r--r-- 1 gerry staff 4666 Jun 2 08:24 /usr/local/Cellar/hadoop/2.8.1/libexec/etc/hadoop/hadoop-env.sh
OK, control+D and then try again with this diff:
diff --git a/Formula/hadoop.rb b/Formula/hadoop.rb
index 5d578a06a3..8e950c3042 100644
--- a/Formula/hadoop.rb
+++ b/Formula/hadoop.rb
@@ -1,7 +1,8 @@
class Hadoop < Formula
desc "Framework for distributed processing of large data sets"
homepage "https://hadoop.apache.org/"
- url "https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz"
+ url "https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz",
+ :using => :nounzip
mirror "https://archive.apache.org/dist/hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz"
sha256 "b5be527578ef2c8565cd435ab4ae71a024cad704f7bf597bd792082703bd77d9"
@@ -10,6 +11,8 @@ class Hadoop < Formula
depends_on :java => "1.7+"
def install
+ system "tar", "xf", "hadoop-2.8.1.tar.gz"
+ cd "hadoop-2.8.1"
rm_f Dir["bin/*.cmd", "sbin/*.cmd", "libexec/*.cmd", "etc/hadoop/*.cmd"]
libexec.install %w[bin sbin libexec share etc]
bin.write_exec_script Dir["#{libexec}/bin/*"]
@@ -17,6 +20,7 @@ class Hadoop < Formula
# But don't make rcc visible, it conflicts with Qt
(bin/"rcc").unlink
+=begin
inreplace "#{libexec}/etc/hadoop/hadoop-env.sh",
"export JAVA_HOME=${JAVA_HOME}",
"export JAVA_HOME=\"$(/usr/libexec/java_home)\""
@@ -26,6 +30,7 @@ class Hadoop < Formula
inreplace "#{libexec}/etc/hadoop/mapred-env.sh",
"# export JAVA_HOME=/home/y/libexec/jdk1.6.0/",
"export JAVA_HOME=\"$(/usr/libexec/java_home)\""
+=end
end
def caveats; <<-EOS.undent
Before I do this, is this what the final file is supposed to look like?
class Hadoop < Formula
desc "Framework for distributed processing of large data sets"
homepage "https://hadoop.apache.org/"
url "https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz",
:using => :nounzip
mirror "https://archive.apache.org/dist/hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz"
sha256 "b5be527578ef2c8565cd435ab4ae71a024cad704f7bf597bd792082703bd77d9"
depends_on :java => "1.7+"
def install
system "tar", "xf", "hadoop-2.8.1.tar.gz"
cd "hadoop-2.8.1"
rm_f Dir["bin/*.cmd", "sbin/*.cmd", "libexec/*.cmd", "etc/hadoop/*.cmd"]
libexec.install %w[bin sbin libexec share etc]
bin.write_exec_script Dir["#{libexec}/bin/*"]
sbin.write_exec_script Dir["#{libexec}/sbin/*"]
# But don't make rcc visible, it conflicts with Qt
(bin/"rcc").unlink
inreplace "#{libexec}/etc/hadoop/hadoop-env.sh",
"export JAVA_HOME=${JAVA_HOME}",
"export JAVA_HOME=\"$(/usr/libexec/java_home)\""
inreplace "#{libexec}/etc/hadoop/mapred-env.sh",
"# export JAVA_HOME=/home/y/libexec/jdk1.6.0/",
"export JAVA_HOME=\"$(/usr/libexec/java_home)\""
end
def caveats; <<-EOS.undent
In Hadoop's config file:
#{libexec}/etc/hadoop/hadoop-env.sh,
#{libexec}/etc/hadoop/mapred-env.sh and
#{libexec}/etc/hadoop/yarn-env.sh
$JAVA_HOME has been set to be the output of:
/usr/libexec/java_home
EOS
end
test do
system bin/"hadoop", "fs", "-ls"
end
end
No you're missing the =begin
and =end
to comment the section out.
It's running now, and a whole lot longer than it has in the past, so that may be a good sign. Please stay tuned.
Seems to have worked:
| => HOMEBREW_NO_AUTO_UPDATE=1 brew upgrade -dv hadoop
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/hadoop.rb
==> Upgrading 1 outdated package, with result:
hadoop 2.8.1
==> Upgrading hadoop
rm /usr/local/bin/container-executor
rm /usr/local/bin/hadoop
rm /usr/local/bin/hdfs
rm /usr/local/bin/mapred
rm /usr/local/bin/test-container-executor
rm /usr/local/bin/yarn
rm /usr/local/sbin/distribute-exclude.sh
rm /usr/local/sbin/hadoop-daemon.sh
rm /usr/local/sbin/hadoop-daemons.sh
rm /usr/local/sbin/hdfs-config.sh
rm /usr/local/sbin/httpfs.sh
rm /usr/local/sbin/kms.sh
rm /usr/local/sbin/mr-jobhistory-daemon.sh
rm /usr/local/sbin/refresh-namenodes.sh
rm /usr/local/sbin/slaves.sh
rm /usr/local/sbin/start-all.sh
rm /usr/local/sbin/start-balancer.sh
rm /usr/local/sbin/start-dfs.sh
rm /usr/local/sbin/start-secure-dns.sh
rm /usr/local/sbin/start-yarn.sh
rm /usr/local/sbin/stop-all.sh
rm /usr/local/sbin/stop-balancer.sh
rm /usr/local/sbin/stop-dfs.sh
rm /usr/local/sbin/stop-secure-dns.sh
rm /usr/local/sbin/stop-yarn.sh
rm /usr/local/sbin/yarn-daemon.sh
rm /usr/local/sbin/yarn-daemons.sh
/usr/bin/sandbox-exec -f /Users/gerry/mytemp/homebrew20170907-12498-19o27r1.sb nice /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/bin/ruby -W0 -I /usr/local/Homebrew/Library/Homebrew -- /usr/local/Homebrew/Library/Homebrew/build.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/hadoop.rb --verbose --debug
/usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FromPathLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/hadoop.rb
==> Downloading https://www.apache.org/dyn/closer.cgi?path=hadoop/common/hadoop-2.8.1/hadoop-2.8.1.tar.gz
Already downloaded: /Users/gerry/Library/Caches/Homebrew/hadoop-2.8.1.tar.gz
==> Verifying hadoop-2.8.1.tar.gz checksum
==> tar xf hadoop-2.8.1.tar.gz
==> Cleaning
Fixing /usr/local/Cellar/hadoop/2.8.1/bin/container-executor permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/bin/hadoop permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/bin/hdfs permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/bin/mapred permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/bin/test-container-executor permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/bin/yarn permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/distribute-exclude.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/hadoop-daemon.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/hadoop-daemons.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/hdfs-config.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/httpfs.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/kms.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/mr-jobhistory-daemon.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/refresh-namenodes.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/slaves.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/start-all.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/start-balancer.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/start-dfs.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/start-secure-dns.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/start-yarn.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/stop-all.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/stop-balancer.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/stop-dfs.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/stop-secure-dns.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/stop-yarn.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/yarn-daemon.sh permissions from 644 to 555
Fixing /usr/local/Cellar/hadoop/2.8.1/sbin/yarn-daemons.sh permissions from 644 to 555
==> Finishing up
ln -s ../Cellar/hadoop/2.8.1/bin/container-executor container-executor
ln -s ../Cellar/hadoop/2.8.1/bin/hadoop hadoop
ln -s ../Cellar/hadoop/2.8.1/bin/hdfs hdfs
ln -s ../Cellar/hadoop/2.8.1/bin/mapred mapred
ln -s ../Cellar/hadoop/2.8.1/bin/test-container-executor test-container-executor
ln -s ../Cellar/hadoop/2.8.1/bin/yarn yarn
ln -s ../Cellar/hadoop/2.8.1/sbin/distribute-exclude.sh distribute-exclude.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/hadoop-daemon.sh hadoop-daemon.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/hadoop-daemons.sh hadoop-daemons.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/hdfs-config.sh hdfs-config.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/httpfs.sh httpfs.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/kms.sh kms.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/mr-jobhistory-daemon.sh mr-jobhistory-daemon.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/refresh-namenodes.sh refresh-namenodes.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/slaves.sh slaves.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/start-all.sh start-all.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/start-balancer.sh start-balancer.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/start-dfs.sh start-dfs.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/start-secure-dns.sh start-secure-dns.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/start-yarn.sh start-yarn.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/stop-all.sh stop-all.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/stop-balancer.sh stop-balancer.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/stop-dfs.sh stop-dfs.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/stop-secure-dns.sh stop-secure-dns.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/stop-yarn.sh stop-yarn.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/yarn-daemon.sh yarn-daemon.sh
ln -s ../Cellar/hadoop/2.8.1/sbin/yarn-daemons.sh yarn-daemons.sh
/usr/bin/sandbox-exec -f /Users/gerry/mytemp/homebrew20170907-13483-1alpxvg.sb nice /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.0.0-p648/bin/ruby -W0 -I /usr/local/Homebrew/Library/Homebrew -- /usr/local/Homebrew/Library/Homebrew/postinstall.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/hadoop.rb -dv
/usr/local/Homebrew/Library/Homebrew/postinstall.rb (Formulary::FromPathLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/hadoop.rb
==> Caveats
In Hadoop's config file:
/usr/local/opt/hadoop/libexec/etc/hadoop/hadoop-env.sh,
/usr/local/opt/hadoop/libexec/etc/hadoop/mapred-env.sh and
/usr/local/opt/hadoop/libexec/etc/hadoop/yarn-env.sh
$JAVA_HOME has been set to be the output of:
/usr/libexec/java_home
==> Summary
🍺 /usr/local/Cellar/hadoop/2.8.1: 25,230 files, 2.1GB, built in 8 minutes 44 seconds
Yeah, so if you want to use that installation, you can manually replace all instances of
export JAVA_HOME=${JAVA_HOME}
with
export JAVA_HOME="$(/usr/libexec/java_home)"
in /usr/local/Cellar/hadoop/2.8.1/libexec/etc/hadoop/hadoop-env.sh
and
# export JAVA_HOME=/home/y/libexec/jdk1.6.0/
with
export JAVA_HOME="$(/usr/libexec/java_home)"
in /usr/local/Cellar/hadoop/2.8.1/libexec/etc/hadoop/yarn-env.sh
and
# export JAVA_HOME=/home/y/libexec/jdk1.6.0/
with
export JAVA_HOME="$(/usr/libexec/java_home)"
in /usr/local/Cellar/hadoop/2.8.1/libexec/etc/hadoop/mapred-env.sh
Something does seem quite wrong with your system, though. Neither the Errno::EINTR: Interrupted system call
nor the Error: cannot load such file -- find
make any sense to me.
Crazy question: do you have any antivirus software or similar installed?
Yes, I do. Why do you think that might trip?
I checked and don't see any security issue flagged in my AV software.
Your system freaks out in two different weird ways with the 2.8.1 tarball but not with the 2.8.0 tarball and I cannot reproduce the issue, so it does seem like something that could be explained by security software actively hating 2.8.1
Please always follow these steps:
brew install
ing one, specific formula (not cask) and not every time you runbrew
? If it's a generalbrew
problem please file this issue at https://github.com/Homebrew/brew/issues/new. If it's abrew cask
problem please file this issue at https://github.com/Homebrew/caskroom/homebrew-cask/newbrew update
and retried your prior step?brew doctor
, fixed all issues and retried your prior step?brew gist-logs <formula>
(where<formula>
is the name of the formula that failed) and included the output link?brew gist-logs
didn't work: ranbrew config
andbrew doctor
and included their output with your issue?Please note we will close your issue without comment if you delete or do not fill out the issue checklist and provide ALL the requested information.
Trying to update Hadoop (already installed) and am getting the following error:
Some potential debugging info: