Closed tillandlauer closed 2 years ago
Hi @tillandlauer, thanks for bringing this up! This is something I've been trying to figure how to best resolve for a while. Installing via brew assumes the user is using a mac and has brew installed, which I could probably write some checks for. Then there's also the issues with root privileges, for installing brew. And of course, we'd need another solution for Linux/Windows users.
I've also explored using zstd installed via conda. But from what I recall, FINEMAP wasn't able to find this installation of ztd and doesn't have an argument to specify where zstd is.
Ideally, zstandard would be included in FINEMAP itself, but I haven't heard of any plans to do this.
So in the end I thought it best to just give user the warning message. But I'll revisit this now and see if I can come up with a better solution. Any suggestions for how to implement this are welcome!
I've also noticed that following the instructions from http://www.christianbenner.com/ to install zstd to solve this issue with FINEMAPv1.4 doesn't seem to work on my Mac either (though this does not appear to be an issue on Linux). This is why I have echolocatoR then default to v1.3 upon failing with v1.4. I've reached out to the FINEMAP author to see if he can provide a fix for this.
Response from FINEMAP author, Christian Benner, who replied a long time ago but just documenting here now for users:
There error message below is saying that you are missing a file in that location. Have you tried to install GCC compiler version 8 using brew install gcc@8? If that doesnโt create the directory and copies the files in there then I would create the directory myself and put a symbolic link.
Following these instructions on my Mac:
brew install gcc@8
However, the output indicates I already have gcc8 installed:
Running `brew update --auto-update`...
==> Auto-updated Homebrew!
Updated 3 taps (homebrew/core, homebrew/cask and homebrew/cask-fonts).
==> New Formulae
burst crytic-compile ghorg linux-headers@5.15 pymol slither-analyzer
cargo-crev dsda-doom git-sync lucky-commit rdb solc-select
censys gcc@11 go@1.18 ouch ripsecrets svt-av1
cql-proxy gcem libvatek purescript-language-server rush-parallel swiftdraw
==> New Casks
aliwangwang decentr font-maple glance-chamburr micromamba playcover-community trunk-io
bookwright filen font-nabla goxel mints qspace-pro virtualbuddy
chromium-gost font-amiri-quran fresh httpie moderndeck texifier yubihsm2-sdk
cider font-cairo-play gittyup imhex nanoem tl-legacy
You have 82 outdated formulae and 3 outdated casks installed.
You can upgrade them with brew upgrade
or list them with brew outdated.
Warning: gcc@8 8.5.0 is already installed and up-to-date.
To reinstall 8.5.0, run:
brew reinstall gcc@8
Just in case there are multiple versions of gcc installed on my computer, I checked which version is being defaulted to in the terminal:
which gcc
# /usr/bin/gcc
/usr/bin/gcc -v
Apple clang version 13.1.6 (clang-1316.0.21.2.5)
Target: x86_64-apple-darwin21.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
Not sure why clang is coming up instead of gcc, but this might be related.
brew reinstall gcc@8
Full output messages:
But this didn't seem to change anything, same issue as before (even when i start a new bash session):
Apple clang version 13.1.6 (clang-1316.0.21.2.5)
Target: x86_64-apple-darwin21.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
Since this issues was first noted, my workaround has been to build in a tryCatch
to any FINEMAP
functions that use the executable. It first tries the latest version (which requires gcc8), but then if that fails it goes back to FINEMAP v1.3.1, which seems to run ok without it.
aha! think i found something regarding the symlinks based on this post:
/usr/local/bin/gcc-8 -v
Using built-in specs.
COLLECT_GCC=/usr/local/bin/gcc-8
COLLECT_LTO_WRAPPER=/usr/local/Cellar/gcc@8/8.5.0/libexec/gcc/x86_64-apple-darwin21/8.5.0/lto-wrapper
Target: x86_64-apple-darwin21
Configured with: ../configure --prefix=/usr/local/Cellar/gcc@8/8.5.0 --libdir=/usr/local/Cellar/gcc@8/8.5.0/lib/gcc/8 --disable-nls --enable-checking=release --enable-languages=c,c++,objc,obj-c++,fortran --program-suffix=-8 --with-gmp=/usr/local/opt/gmp --with-mpfr=/usr/local/opt/mpfr --with-mpc=/usr/local/opt/libmpc --with-isl=/usr/local/opt/isl --with-pkgversion='Homebrew GCC 8.5.0' --with-bugurl=https://github.com/Homebrew/homebrew-core/issues --build=x86_64-apple-darwin21 --with-system-zlib --disable-multilib --with-native-system-header-dir=/usr/include --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk --without-build-config
Thread model: posix
gcc version 8.5.0 (Homebrew GCC 8.5.0)
This does indeed seem to be the executable I want. Now i need to find a way to redirect the symlink to this version (preferably from within R).
In fact, it seems I have multiple versions of gcc installed on my Mac:
bash-3.2$ echo $PATH
/opt/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/schilder/Library/Python/3.8/bin:/opt/X11/bin:/Library/Apple/usr/bin:/Library/TeX/texbin
bash-3.2$ /usr/local/bin/gcc
gcc gcc-12 gcc-8 gcc-ar-12 gcc-ar-8 gcc-nm-12 gcc-nm-8 gcc-ranlib-12 gcc-ranlib-8
ln -sf /usr/local/bin/gcc-8 /usr/local/bin/gcc
After starting a new bash session, the right gcc is now being referenced!
Now to figure out how to do this within R in a way that communicates with FINEMAP.
Even though I can see the right version of gcc is indeed being referred to....
out <- R.utils::createLink(
link="/usr/local/bin/gcc",
target="/usr/local/bin/gcc-8",
overwrite=TRUE)
system("gcc -v")
Using built-in specs.
COLLECT_GCC=/usr/local/bin/gcc
COLLECT_LTO_WRAPPER=/usr/local/Cellar/gcc@8/8.5.0/libexec/gcc/x86_64-apple-darwin21/8.5.0/lto-wrapper
Target: x86_64-apple-darwin21
Configured with: ../configure --prefix=/usr/local/Cellar/gcc@8/8.5.0 --libdir=/usr/local/Cellar/gcc@8/8.5.0/lib/gcc/8 --disable-nls --enable-checking=release --enable-languages=c,c++,objc,obj-c++,fortran --program-suffix=-8 --with-gmp=/usr/local/opt/gmp --with-mpfr=/usr/local/opt/mpfr --with-mpc=/usr/local/opt/libmpc --with-isl=/usr/local/opt/isl --with-pkgversion='Homebrew GCC 8.5.0' --with-bugurl=https://github.com/Homebrew/homebrew-core/issues --build=x86_64-apple-darwin21 --with-system-zlib --disable-multilib --with-native-system-header-dir=/usr/include --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk --without-build-config
Thread model: posix
gcc version 8.5.0 (Homebrew GCC 8.5.0)
FINEMAP doesn't' seem to be able to recognize it. Even including the softlink creation within the same command doesn't help:
system("/Users/schilder/Library/Caches/org.R-project.R/R/echofinemap/FINEMAP/finemap_v1.4.1_MacOSX/finemap_v1.4.1_MacOSX")
## or
system("ln -sF /usr/local/bin/gcc-8 /usr/local/bin/gcc & /Users/schilder/Library/Caches/org.R-project.R/R/echofinemap/FINEMAP/finemap_v1.4.1_MacOSX/finemap_v1.4.1_MacOSX")
dyld[89574]: Library not loaded: /usr/local/opt/gcc/lib/gcc/10/libgfortran.5.dylib
Referenced from: /Users/schilder/Library/Caches/org.R-project.R/R/echofinemap/FINEMAP/finemap_v1.4.1_MacOSX/finemap_v1.4.1_MacOSX
Reason: tried: '/usr/local/opt/gcc/lib/gcc/10/libgfortran.5.dylib' (no such file), '/usr/local/lib/libgfortran.5.dylib' (no such file), '/usr/lib/libgfortran.5.dylib' (no such file)
sh: line 1: 89574 Abort trap: 6 /Users/schilder/Library/Caches/org.R-project.R/R/echofinemap/FINEMAP/finemap_v1.4.1_MacOSX/finemap_v1.4.1_MacOSX
It doesn't even seem to be R related, as FINEMAP has this same issue when called from the terminal. Maybe it has something to do with where it's cashed?
GOT IT!!
Ok, so several things needed to be in place:
gcc
must be installed (via brew), though any version seems to work fine (e.g. 7,8,10): brew install gcc@10
gcc
to override any clang-based versions (e.g. that used by xcode on Macs).libomp
must be installed (this took me a while to figure out that this was a totally separate software): brew install libomp
zstd
must be installed: brew install zstd
I've created an internal function in echofinemap
called setup_gcc
which automates the symlink steps by searching for an appropriate gcc
version automatically. If none can be found, it instructs the user to install one using brew
.
https://github.com/RajLabMSSM/echofinemap/blob/efbb2ef08d1e9d7b700bcbf92445917813ab364c/R/setup_gcc.R
Because it overwrite any preexisting symlinks, setup_gcc
allows for easy switching between different versions of gcc
installed on the same machine at once.
Now FINEMAP v1.4.1 successfully runs all the way through!!
Loading required namespace: MungeSumstats
Preparing sample size column (N).
Computing effective sample size using the LDSC method:
Neff = (N_CAS+N_CON) * (N_CAS/(N_CAS+N_CON)) / mean((N_CAS/(N_CAS+N_CON))[(N_CAS+N_CON)==max(N_CAS+N_CON)]))
+ Mapping colnames from MungeSumstats ==> echolocatoR
+ Subsetting LD matrix and dat to common SNPs...
Removing unnamed rows/cols
Replacing NAs with 0
+ LD_matrix = 95 SNPs.
+ dat = 95 SNPs.
+ 95 SNPs in common.
Converting obj to sparseMatrix.
Constructing master file.
Constructing data.z file.
Constructing data.ld file.
Using gcc version: 8
FINEMAP path: /Users/schilder/Library/Caches/org.R-project.R/R/echofinemap/FINEMAP/finemap_v1.4.1_MacOSX/finemap_v1.4.1_MacOSX
Inferred FINEMAP version: 1.4.1
Running FINEMAP.
cd .../BST1 &&
.../finemap_v1.4.1_MacOSX
--sss
--in-files .../master
--log
--n-causal-snps 5
|--------------------------------------|
| Welcome to FINEMAP v1.4.1 |
| |
| (c) 2015-2022 University of Helsinki |
| |
| Help : |
| - ./finemap --help |
| - www.finemap.me |
| - www.christianbenner.com |
| |
| Contact : |
| - finemap@christianbenner.com |
| - matti.pirinen@helsinki.fi |
|--------------------------------------|
--------
SETTINGS
--------
- dataset : all
- corr-config : 0.95
- n-causal-snps : 5
- n-configs-top : 50000
- n-conv-sss : 100
- n-iter : 100000
- n-threads : 1
- prior-k0 : 0
- prior-std : 0.05
- prob-conv-sss-tol : 0.001
- prob-cred-set : 0.95
------------
FINE-MAPPING (1/1)
------------
- GWAS summary stats : FINEMAP/data.z
- SNP correlations : FINEMAP/data.ld
- Causal SNP stats : FINEMAP/data.snp
- Causal configurations : FINEMAP/data.config
- Credible sets : FINEMAP/data.cred
- Log file : FINEMAP/data.log_sss
- Reading input : done!
- Number of GWAS samples : 1474097
- Number of SNPs : 95
- Prior-Pr(# of causal SNPs is k) :
(0 -> 0)
1 -> 0.584
2 -> 0.292
3 -> 0.0963
4 -> 0.0236
5 -> 0.00456
- 1737 configurations evaluated (0.105/100%) : converged after 105 iterations
- Computing causal SNP statistics : done!
- Regional SNP heritability : 0.00296 (SD: 8.85e-05 ; 95% CI: [0.00279,0.00313])
- Log10-BF of >= one causal SNP : 931
- Post-expected # of causal SNPs : 5
- Post-Pr(# of causal SNPs is k) :
(0 -> 0)
1 -> 0
2 -> 0
3 -> 0
4 -> 2.61e-49
5 -> 1
- Writing output : done!
- Run time : 0 hours, 0 minutes, 0 seconds
1 data.cred* file(s) found in the same subfolder.
Selected file based on postPr_k: data.cred5
Importing conditional probabilities (.cred file).
Importing marginal probabilities (.snp file).
Importing configuration probabilities (.config file).
platform x86_64-apple-darwin17.0
arch x86_64
os darwin17.0
system x86_64, darwin17.0
status
major 4
minor 2.1
year 2022
month 06
day 23
svn rev 82513
language R
version.string R version 4.2.1 (2022-06-23)
nickname Funny-Looking Kid
apologies for re-opening the issue:
I am on a Mac arm64 architecture - and the procedure your have outlined here doesn't seem to be working for me. I was wondering if there is an equivalent for arm64 case?
apologies for re-opening the issue:
I am on a Mac arm64 architecture - and the procedure your have outlined here doesn't seem to be working for me. I was wondering if there is an equivalent for arm64 case?
Hi @asoroosh, I'll try to look into it
Could you send me the specific specs on your Mac? Should be findable via System Setting --> System Report --> Hardware
Hi Brian - thanks for your reply. See the spec below:
FINEMAP won't run on a Mac unless Zstandard is installed (error message:
dyld: Library not loaded: /usr/local/lib/libzstd.1.dylib
), see: http://www.christianbenner.comMac OSX users: If you see dyld: Library not loaded: /usr/local/lib/libzstd.1.dylib, install Zstandard.
. The Zstandard lib (https://facebook.github.io/zstd/) can be installed in macOS usingbrew install zstd
. Please consider adding this library to the install routine!