Broken pipe error while installing on Mac OS

[serhan-gul]

I'm trying to install scanner on OS X High Sierra (10.13.6) using Homebrew. However, during the installation I get this error:

[ 78%] Built target engine
make: *** [all] Error 2
/usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `write': Broken pipe (Errno::EPIPE)
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `puts'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `rescue in block (3 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:31:in `block (3 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:30:in `fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:30:in `block (2 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:27:in `open'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:27:in `block in safe_fork'
    from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/tmpdir.rb:89:in `mktmpdir'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:26:in `safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/formula_installer.rb:719:in `build'
    from /usr/local/Homebrew/Library/Homebrew/formula_installer.rb:311:in `install'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:321:in `install_formula'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:253:in `block in install'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:251:in `each'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:251:in `install'
    from /usr/local/Homebrew/Library/Homebrew/brew.rb:89:in `<main>'

What can be the cause of this error? Anyone experiences something similar? Thanks.

[willcrichton]

I just tried installing fresh on High Sierra with no issues. That seems like possibly just a transient error--can you try running it again and see if it works?

[serhan-gul]

I tried restarting the terminal and the operating system then running again. Still got the same error. I couldn't find much about the cause of this error by googling. It might be a problem related to Homebrew, I think. brew -v tells me that I'm using the version 1.7.4, which is the latest version. I will try to downgrade and try again.

Meanwhile, I managed to install using Docker; the first tutorial example works fine (although I had to increase the assigned memory from 2GB to 8GB; otherwise the process is killed).

[fpoms]

Hi @serhannn! Any updates on your end? Let us know if this is still an issue.

[serhan-gul]

The problem still persists but I suspect that it might be related to some bug in homebrew after browsing this issue. I tried replacing the file /usr/local/Homebrew/Library/Homebrew/utils/fork.rb with the updated version but it didn't help.

I also tried installing some other packages via homebrew. It worked without a problem.

[darrylyoung]

I'm also having this problem on a clean install of macOS High Sierra. All other packages seem to be fine but I get the issue with watchman.

[fpoms]

I just brew upgrade && brew update and now have version: Homebrew 1.7.4-54-g2635160

My OS X version is: 10.13.1

I was able to run brew reinstall --verbose --debug scanner without any issues.

Can you list your OS version and homebrew version string? Perhaps try reinstalling one more time as well.

[claui]

I can reproduce the issue on a 10.13.6 VM.

$ brew tap scanner-research/homebrew-scanner
[…]
$ brew install scanner
[…]
==> Installing scanner-research/scanner/scanner
==> Downloading https://github.com/scanner-research/scanner/archive/v0.2.18.tar.
==> Downloading from https://codeload.github.com/scanner-research/scanner/tar.gz
          -=O=#     #    #     #                                              
==> bash deps.sh -a -ng --prefix /usr/local/Cellar/scanner/0.2.18 --with-ffmpeg 
==> cmake .. -DCMAKE_C_FLAGS_RELEASE=-DNDEBUG -DCMAKE_CXX_FLAGS_RELEASE=-DNDEBUG
==> make
==> python3 setup.py bdist_wheel
==> CMAKE_PREFIX_PATH= PKG_CONFIG_PATH= pip3 install --prefix=/usr/local/Cellar/
==> pip3 install --prefix=/usr/local/Cellar/scanner/0.2.18/libexec dist/*
Last 15 lines from /Users/vagrant/Library/Logs/Homebrew/scanner/06.*:
Collecting testing.common.database (from testing.postgresql==1.3.0->scannerpy==0.2.18)
  Downloading https://files.pythonhosted.org/packages/a7/1a/ca1c39544ed92fa8ea121ff3bf05bb4838520c498942054235ebc4a83b36/testing.common.database-2.0.3-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from protobuf==3.5.1->scannerpy==0.2.18) (40.2.0)
Building wheels for collected packages: psycopg2
  Running setup.py bdist_wheel for psycopg2: started
  Running setup.py bdist_wheel for psycopg2: finished with status 'done'
  Stored in directory: /private/tmp/scanner-20180916-13131-m4dhq8/scanner-0.2.18/.brew_home/Library/Caches/pip/wheels/43/ff/71/a0b0d6dbf71f912b95cf18101bca206b40eed5086d8fdb4ed9
Successfully built psycopg2
Installing collected packages: cloudpickle, scipy, psycopg2, grpcio, tqdm, pg8000, testing.common.database, testing.postgresql, enum34, protobuf, toml, scannerpy
  The script tqdm is installed in '/usr/local/Cellar/scanner/0.2.18/libexec/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  Found existing installation: protobuf 3.6.0
    Uninstalling protobuf-3.6.0:
Could not install packages due to an EnvironmentError: [Errno 1] Operation not permitted: '/usr/local/Cellar/hwang/0.3.3/libexec/lib/python3.7/site-packages/google/protobuf/__init__.py'

/usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `write': Broken pipe (Errno::EPIPE)
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `puts'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `rescue in block (3 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:31:in `block (3 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:30:in `fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:30:in `block (2 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:27:in `open'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:27:in `block in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/2.3.0/tmpdir.rb:89:in `mktmpdir'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:26:in `safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/formula_installer.rb:719:in `build'
    from /usr/local/Homebrew/Library/Homebrew/formula_installer.rb:311:in `install'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:321:in `install_formula'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:253:in `block in install'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:251:in `each'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:251:in `install'
    from /usr/local/Homebrew/Library/Homebrew/brew.rb:89:in `<main>'

If reporting this issue please do so at (not Homebrew/brew or Homebrew/core):
https://github.com/scanner-research/homebrew-scanner/issues

[fpoms]

Hi @claui, can you run the same command with debug flags enabled and paste the output here?

brew install scanner --verbose --debug

For some reason I can't reproduce this on my machine.

[claui]

@apoms https://gist.github.com/claui/cd833412b7d258de902ec1b3f51d1f67

[fpoms]

Seems to be an issue downloading a file from AWS S3. Can you try running:

wget https://github.com/halide/Halide/releases/download/release_2018_02_15/halide-mac-64-trunk-46d8e9e0cdae456489f1eddfd6d829956fc3c843.tgz

and showing the output here?

[claui]

Hmm, that worked flawlessly:

--2018-09-16 22:27:08--  https://github.com/halide/Halide/releases/download/release_2018_02_15/halide-mac-64-trunk-46d8e9e0cdae456489f1eddfd6d829956fc3c843.tgz
Resolving github.com (github.com)... 192.30.253.112, 192.30.253.113
Connecting to github.com (github.com)|192.30.253.112|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/5241312/5ceed4f8-1240-11e8-9b97-460c0d4ef6f6?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180916%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180916T202708Z&X-Amz-Expires=300&X-Amz-Signature=a35e90068eefb804dd9e8ea939bf1f409f0143938cc0fa36c3ec5f3cf8e22807&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dhalide-mac-64-trunk-46d8e9e0cdae456489f1eddfd6d829956fc3c843.tgz&response-content-type=application%2Foctet-stream [following]
--2018-09-16 22:27:08--  https://github-production-release-asset-2e65be.s3.amazonaws.com/5241312/5ceed4f8-1240-11e8-9b97-460c0d4ef6f6?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180916%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180916T202708Z&X-Amz-Expires=300&X-Amz-Signature=a35e90068eefb804dd9e8ea939bf1f409f0143938cc0fa36c3ec5f3cf8e22807&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dhalide-mac-64-trunk-46d8e9e0cdae456489f1eddfd6d829956fc3c843.tgz&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.0.0
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.0.0|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 63478710 (61M) [application/octet-stream]
Saving to: ‘halide-mac-64-trunk-46d8e9e0cdae456489f1eddfd6d829956fc3c843.tgz’

halide-mac-64-trunk-46d8e 100%[=====================================>]  60,54M  1,72MB/s    in 33s     

[fpoms]

Could you try running it a few times to see if you can reproduce the issue? I'm adding some code to the script to retry in the case of 403 errors, but it would be good if you could reproduce outside of the homebrew install script.

[claui]

Another brew install scanner --verbose --debug invocation that failed:

• Output
• Log file ~/Library/Logs/Homebrew/scanner/06.* (yes, the file name is literally 06.*)

I wonder whether the errors I’ve encountered all have the same root cause. Those errors seem so random to me, with hardly anything in common with each other.

I’ll keep trying to reproduce outside of the install script.

[fpoms]

@claui Ok, I think I figured out the issue. The problem is that brew does not let you touch system installed python packages via pip inside the build script. This means that if Scanner needs a different version of protobuf than what is installed, then it will fail. I've changed the script to not install the python dependencies and instead inform the user to run:

brew install scanner
pip3 install scannerpy

Try that out and let me know if it works! I suggest first running:

brew uninstall hwang
brew uninstall scanner

[claui]

@apoms This reproducibly fails with an SHA mismatch.

Error: An exception occured within a build process:
  ChecksumMismatchError: SHA256 mismatch
Expected: 14e5a2515eb85251a4395da31f960b62be54643d205b378a2ccef3cad1791c05
Actual: a7c674ade67660ef7681333aac8a8d9ec9a884dc6afb730eb612a495e8a3734c
Archive: /Users/vagrant/Library/Caches/Homebrew/downloads/a74a0cf2757b62fb4d24224173cc8217a99900447144933664ed9fe665187b05--scanner-0.2.20.tar.gz
To retry an incomplete download, remove the file above.

[fpoms]

Apologies, I just pushed a change to update an issue with our automated build which changed the SHA. Will left you know when that’s fixed. On Tue, Sep 18, 2018 at 12:09 AM Claudia notifications@github.com wrote:

@apoms https://github.com/apoms This reproducibly fails with an SHA mismatch.

Error: An exception occured within a build process: ChecksumMismatchError: SHA256 mismatch Expected: 14e5a2515eb85251a4395da31f960b62be54643d205b378a2ccef3cad1791c05 Actual: a7c674ade67660ef7681333aac8a8d9ec9a884dc6afb730eb612a495e8a3734c Archive: /Users/vagrant/Library/Caches/Homebrew/downloads/a74a0cf2757b62fb4d24224173cc8217a99900447144933664ed9fe665187b05--scanner-0.2.20.tar.gz To retry an incomplete download, remove the file above.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/scanner-research/scanner/issues/225#issuecomment-422280264, or mute the thread https://github.com/notifications/unsubscribe-auth/AAICj-uQdT71SIa53I25rw0exI4CjNhKks5ucJxFgaJpZM4WjmDZ .

[darrylyoung]

This is what I'm getting when trying to install watchman. I guess this is the same issue some of you are seeing when installed other packages.

➜  ~ brew install watchman
==> Downloading https://homebrew.bintray.com/bottles/watchman-4.9.0.mojave.bottle.tar.gz
Already downloaded: /Users/dyoung/Library/Caches/Homebrew/downloads/34f18f3641cfc75025fdcf4c174cff8911bf2bf72fd933a8f90465149e997d71--watchman-4.9.0.mojave.bottle.tar.gz
==> Pouring watchman-4.9.0.mojave.bottle.tar.gz
/usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `write': Broken pipe (Errno::EPIPE)
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `puts'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `rescue in block (3 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:31:in `block (3 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:30:in `fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:30:in `block (2 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:27:in `open'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:27:in `block in safe_fork'
    from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/tmpdir.rb:89:in `mktmpdir'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:26:in `safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/formula_installer.rb:886:in `post_install'
    from /usr/local/Homebrew/Library/Homebrew/formula_installer.rb:615:in `finish'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:322:in `install_formula'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:253:in `block in install'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:251:in `each'
    from /usr/local/Homebrew/Library/Homebrew/cmd/install.rb:251:in `install'
    from /usr/local/Homebrew/Library/Homebrew/brew.rb:89:in `<main>'
Warning: The post-install step did not complete successfully
You can try again using `brew postinstall watchman`
==> Summary
🍺  /usr/local/Cellar/watchman/4.9.0: 23 files, 2.1MB

➜  ~ brew postinstall watchman
==> Postinstalling watchman
/usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `write': Broken pipe (Errno::EPIPE)
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `puts'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:49:in `rescue in block (3 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:31:in `block (3 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:30:in `fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:30:in `block (2 levels) in safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:27:in `open'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:27:in `block in safe_fork'
    from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/tmpdir.rb:89:in `mktmpdir'
    from /usr/local/Homebrew/Library/Homebrew/utils/fork.rb:26:in `safe_fork'
    from /usr/local/Homebrew/Library/Homebrew/formula_installer.rb:886:in `post_install'
    from /usr/local/Homebrew/Library/Homebrew/cmd/postinstall.rb:14:in `block in postinstall'
    from /usr/local/Homebrew/Library/Homebrew/cmd/postinstall.rb:11:in `each'
    from /usr/local/Homebrew/Library/Homebrew/cmd/postinstall.rb:11:in `postinstall'
    from /usr/local/Homebrew/Library/Homebrew/brew.rb:89:in `<main>'
Warning: The post-install step did not complete successfully
You can try again using `brew postinstall watchman`

[serhan-gul]

@claui Ok, I think I figured out the issue. The problem is that brew does not let you touch system installed python packages via pip inside the build script. This means that if Scanner needs a different version of protobuf than what is installed, then it will fail. I've changed the script to not install the python dependencies and instead inform the user to run:

brew install scanner
pip3 install scannerpy

Try that out and let me know if it works! I suggest first running:

brew uninstall hwang
brew uninstall scanner

@apoms Thanks for dealing with this! I managed to get rid of the broken pipe error. Running brew install scanner --verbose --debug showed me that there were some mismatch about the OpenCV libraries, so I had to make a clean install of OpenCV.

Then, I could install scanner using brew install scanner and then the python dependencies with pip3 install scannerpy. Now when I start python and do import scannerpy, I get an error related to protobuf:

OSError: dlopen(/Users/guel/miniconda3/lib/python3.6/site-packages/scannerpy/lib/libscanner.dylib, 6): Library not loaded: /usr/local/opt/protobuf/lib/libprotobuf.15.dylib
  Referenced from: /Users/guel/miniconda3/lib/python3.6/site-packages/scannerpy/lib/libscanner.dylib
  Reason: image not found 

Listing the directory show sthat I have /usr/local/opt/protobuf/lib/libprotobuf.17.dylib but not libprotobuf.15.dylib as required by Scanner. Is anybody else experiencing that too? Any idea how to get the right version of libprotobuf?

[fpoms]

@serhannn Did you update your protobuf after installing scanner? Scanner builds against your local protobuf install so if you change that it needs to be reinstalled. I would just try rerunning brew reinstall scanner.

[fpoms]

@darrylyoung I was able to install that package without a problem. Maybe try running brew install --verbose --debug watchman?

[fpoms]

@claui Ok, everything should be in a stable state now and in the future. Please try the installation out and let me know if it works.

[woodruffw]

https://github.com/Homebrew/brew/pull/4853 should fix this. The underlying issue was that fetch/build errors were being masked.

[fpoms]

@claui @darrylyoung @serhannn Is this still an issue? If not, let me know so I can close.

[darrylyoung]

@apoms I'm no longer having the issue. Thanks for checking in.

[claui]

@apoms I get an error when installing the hwang dependency. It’s probably unrelated to this issue but it keeps me from checking whether brew install scanner works.

[claui]

https://github.com/scanner-research/hwang/issues/8

[serhan-gul]

@apoms Thanks for checking! I managed to install by brew install scannerand pip3 install scannerpynow and run a basic example. I see @claui continues with another issue so I'm closing this one.