JayDDee / cpuminer-opt

Optimized multi algo CPU miner
Other
774 stars 545 forks source link

OSX Build #123

Closed matthewjumpsoffbuildings closed 6 years ago

matthewjumpsoffbuildings commented 6 years ago

I have managed to successfully build tpruvots cpuminer-multi on OSX using the following

brew install automake openssl zlib curl jansson make bla bla bla
./autogen.sh
chmod +x nomacro.pl && cd asm && ../nomacro.pl && cd .. && ./nomacro.pl
./configure CFLAGS="-march=native -O2 -Ofast -DUSE_ASM -pg -flto" --with-crypto=/usr/local/opt/openssl  --with-curl=/usr/local/opt/curl
perl -p -i -e "s/#if \(WINDOWS\)/#define ASM 0\n#if (WINDOWS)/g" algo/neoscrypt.c
make

however when i try and run the same compile on your cpuminer-opt, it errors

algo/hodl/hodl-gate.c:12:1: error: unknown type name 'pthread_barrier_t'
pthread_barrier_t hodl_barrier;
^
algo/hodl/hodl-gate.c:165:4: warning: implicit declaration of function
      'pthread_barrier_wait' is invalid in C99 [-Wimplicit-function-declaration]
   pthread_barrier_wait( &hodl_barrier );
   ^
algo/hodl/hodl-gate.c:201:3: warning: implicit declaration of function
      'pthread_barrier_init' is invalid in C99 [-Wimplicit-function-declaration]
  pthread_barrier_init( &hodl_barrier, NULL, opt_n_threads );
  ^
2 warnings and 1 error generated.
make[2]: *** [algo/hodl/cpuminer-hodl-gate.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

any chance you would know what this error is and how i can get cpuminer-opt compiling on my osx machine?

matthewjumpsoffbuildings commented 6 years ago

I just saw in your readme you mentioned mac OSX is not supported, but I feel like its so close to being able to work... if you have any advice it would be much appreciated.

For example is there a relatively simple way to drop certain algos from the compile? It seems like the issue is the hodl algo, before failing on hodl it got thru a lot of other algos with no issue... if I could disable hodl somehow it might work?

JayDDee commented 6 years ago

It's very easy to delete most algos. Simply delete the hodl directory, remove the files from Makefile.am and remove the call to register_hodl_algo in algo-gate-api.c.

But honestly you aren't close to compiling it yet, expect a lot more issues.

I don't know how much I can help but I'm curious how far you can get.

matthewjumpsoffbuildings commented 6 years ago

Ok thanks for the advice, I guess I'll try it and if any more algos give me issues I'll keep deleting till i have nothin left :)

matthewjumpsoffbuildings commented 6 years ago

Ok the next algo that errored is simd, i see the simd directory but I cant work out what algo to delete from the Makefile.am and algo-gate-api.c?

matthewjumpsoffbuildings commented 6 years ago

Hmm ok just to test it, i tried commenting out everything but the yescrypt algos in algo-gate-api.c, as well as any algo c files in Makefile.am, just to see if i can compile it with one algo, still erroring.

Now its doing this

undef: _sha256d
Undefined symbols for architecture x86_64:
  "_sha256d", referenced from:
      _sha256d_gen_merkle_root in lto.o
      _parse_arg in lto.o
      _gbt_work_decode in lto.o
     (maybe you meant: _sha256d_ms_8way, _sha256d_ms_4way , _sha256d_gen_merkle_root )
ld: symbol(s) not found for architecture x86_64

perhaps i went too far with my commenting out :/

matthewjumpsoffbuildings commented 6 years ago

Well! I just left in scrypt, sha, skein and yescrypt and it compiled! Its running now, and successfully submitting shares to a WAVI pool! I might go thru now and try and re-enable other algos one by one to see what works and what doesnt.

Thanks for the help :)

JayDDee commented 6 years ago

I left this issue open presuming you would continue reporting progress and problems. The sha256d error looks like a problem with USE_ASM, the configure messages might provide a clue. That problem will affect most algos

b-seite commented 6 years ago

hey @matthewjumpsoffbuildings i found the solution for your initial Problem:

here: http://blog.albertarmea.com/post/47089939939/using-pthreadbarrier-on-mac-os-x i added the codpiece into hodlgate.c and the error is gone.

but i instantly get another error from the hodl algo: algo/hodl/sha512_avx.c:205:5: error: cannot convert between scalar type 'uint64_t' (aka 'unsigned long long') and vector type '__m128i' (vector of 2 'long long' values) as implicit conversion would cause truncation R512_0; R512_0; ^ algo/hodl/sha512_avx.c:164:5: note: expanded from macro 'R512_0' ROUND512_0_TO_15(a, b, c, d, e, f, g, h); \ ^ algo/hodl/sha512_avx.c:156:5: note: expanded from macro 'ROUND512_0_TO_15' ROUND512(a,b,c,d,e,f,g,h) ^ algo/hodl/sha512_avx.c:145:62: note: expanded from macro 'ROUND512' T0 += (h[0]) + SIGMA2(e[0]) + Ch((e[0]), (f[0]), (g[0])) + k[i]; \

anyone any idea?

JayDDee commented 6 years ago

@b-seite

Thanks for the input. The latest error you see looks like a compiler issue. In certain circumstances mixing scalar and vector arguments in an expression is allowed but your compiler won't allow it so the scalar arguments must be converted to vectors. The offending variable is k[] and references need to be wrapped as follows:

define ROUND512(a,b,c,d,e,f,g,h) \

T0 += (h[0]) + SIGMA2(e[0]) + Ch((e[0]), (f[0]), (g[0])) + k[i]; \
T1 += (h[1]) + SIGMA2(e[1]) + Ch((e[1]), (f[1]), (g[1])) + k[i]; \
(d[0]) += T0; \
(d[1]) += T1; \
(h[0]) = T0 + SIGMA1(a[0]) + Maj((a[0]), (b[0]), (c[0])); \
(h[1]) = T1 + SIGMA1(a[1]) + Maj((a[1]), (b[1]), (c[1])); \
i++

define ROUND512(a,b,c,d,e,f,g,h) \

T0 += (h[0]) + SIGMA2(e[0]) + Ch((e[0]), (f[0]), (g[0])) + _mm_set1_epi64x( k[i] ); \
T1 += (h[1]) + SIGMA2(e[1]) + Ch((e[1]), (f[1]), (g[1])) + _mm_set1_epi64x( k[i] ); \
(d[0]) += T0; \
(d[1]) += T1; \
(h[0]) = T0 + SIGMA1(a[0]) + Maj((a[0]), (b[0]), (c[0])); \
(h[1]) = T1 + SIGMA1(a[1]) + Maj((a[1]), (b[1]), (c[1])); \
i++

A similar change needs to be made in the avx2 file using _mm256_set1_epi64x.

b-seite commented 6 years ago

Yes great that helped a lot.

The next Error is on algo simd:

`algo/simd/vector.c:129:3: error: use of unknown builtin '__builtin_ia32_pcmpgtw128' [-Wimplicit-function-declaration] DO_REDUCE_FULL_S(0); ^ algo/simd/vector.c:56:12: note: expanded from macro 'DO_REDUCE_FULL_S' X(i) = EXTRA_REDUCE_S(X(i)); \ ^ algo/simd/vector.c:42:32: note: expanded from macro 'EXTRA_REDUCE_S' v16_sub(x, v16_and(V257.v16, v16_cmp(x, V128.v16))) ^ algo/simd/vector.h:92:22: note: expanded from macro 'v16_cmp'

define v16_cmp __builtin_ia32_pcmpgtw128

                 ^

algo/simd/vector.c:129:3: note: did you mean '__builtin_ia32_pcmpgtw'? algo/simd/vector.c:56:12: note: expanded from macro 'DO_REDUCE_FULL_S' X(i) = EXTRA_REDUCE_S(X(i)); \ ^ algo/simd/vector.c:42:32: note: expanded from macro 'EXTRA_REDUCE_S' v16_sub(x, v16_and(V257.v16, v16_cmp(x, V128.v16))) ^ algo/simd/vector.h:92:22: note: expanded from macro 'v16_cmp'

define v16_cmp __builtin_ia32_pcmpgtw128

                 ^

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.1.0/include/mmintrin.h:1263:19: note: 'builtin_ia32_pcmpgtw' declared here return (m64)builtin_ia32_pcmpgtw((v4hi)m1, (v4hi)__m2); `

there are some more conversion errors also but i think now i can get rid of them on my own

JayDDee commented 6 years ago

This looks like another compiler issue. This part is interesting: algo/simd/vector.c:129:3: note: did you mean '__builtin_ia32_pcmpgtw'?

PCMPGTW is a valid SIMD (not to be confused with the algo of the same name) instruction. I don't know about all this __builtin_ia32 stuff in this code, it seems to be redundant with the SIMD intrinsics defined here: https://software.intel.com/sites/landingpage/IntrinsicsGuide/.

Note: code formatting is not working correctly, doesn't recongnize line breaks, hard to read.

b-seite commented 6 years ago

I did it the hard way: In algo/simd/vector.c i deleted all Lines from Using GCC vector extensions up to Using MSVC/ICC vector instrinsics so the compiler HAS to use the ICC vector instrincts. Now it compiles a lot more. I don't know if that is a solution just for me but maybe we find a better solution than deleting Lines of Code. Maybe with an IF condition, asking for osx?
Next Error is here:

`:5:6: error: unknown token in expression mov [%rsi + 0], eax; ^

:6:6: error: unknown token in expression mov [%rsi + 4], ebx; ^ :7:6: error: unknown token in expression mov [%rsi + 8], ecx; ^ :8:6: error: unknown token in expression mov [%rsi + 12], edx; ^ LLVM ERROR: Error parsing inline asm clang: error: linker command failed with exit code 1 (use -v to see invocation) ` but i think that this might be a ./nomacro.pl issue?
JayDDee commented 6 years ago

I have no idea how to solve the asm problem.

Clang could ultimately be the show stopper, there could be GCC specific code anywhere because cpuminer-opt is a Frankenstein, with code imported from many miners from many developpers. Much of it is specifically for GCC. If you can use GCC you might have better luck.

JayDDee commented 6 years ago

This issue seems to be stalled, there is no point in keeping it open.

lordgamegenie commented 6 years ago

There really needs to be more focus on a Mac OS X version. I have a decent i5 MBP that I know could get better hashrate (from cpuminer-multi) because I seen it on my i5 DELL equivalent using cpuminer-opt. As it looks currently, the only feasible way around this barrier for Mac users would be to install Bootcamp and Windows, and then you will be able to utilize cpuminer-opt.

juanpc2018 commented 6 years ago

i was a Windows8,1 hard core fan, but W10 looks weird, i like more OSX Sierra, a 8700K is 100% faster than i7-970 / x5670 / w3670 "2nd gen i7 for x58 lga1366" or i7-970 / x5670 / w3670 "2nd gen i7 for x58 lga1366" are 50% a 8700k

OSX GPU always has lower framerates than Windows in benchmarks, like Windows is cheating, or having fake frames, OSX has Quartz Extreme that makes graphics look very fluild, Windows graphics feel inferior. but has higher scores in most benchmarks, the old quality vs. quantity dilema.

in cinebench CPU test seems similar to Windows, but cant be sure, i havet tested the exact same cpu, similar, but Not the same, i7-970 vs x5675 Osx seems better.

Linux running Wine, running WinRAR benchmark was inferior, there is Not a Native GUI for Linux version of WinRAR, CLI does Not have benchmark, last time i tested. WinRAR has a problem in windows with more than 32-cores, all cores run at 50%. 7-zip is better code, but gui is lifeless, that why i like more WinRAR, im more SW GUI oriented.

tested linux for a while, but GUI is worse than windows, ubuntu is a clone of osx, kubuntu a clone of windows, has the Unix philosophy limitations, like Not able to use NVIDIA and AMD cards at the same time, or with a differnt kernel, last time i tested, havent tested OSX with different video cards but seems does Not have the kernel issue. ext4 file system was faster than NTFS but loses information if moving files and has a power loss,"less secure". Windows has some weird limitations, takes too much time to boot, & does Not boot from USB3.0 OSX feels it boots faster in USB2.0 than Windows without Instant start, rapid start, OSX seems less bloated or more efficient. but OSX laptops are not as good as Desktops, most laptops fail with the dual video card, needs the igx software to limit the video card and prevent automatic switching..

FreeBSD is the open source brother of OSX & PlayStation4, claims to run Linux software faster than Linux in x86_64 but looking aarch64 videos seems needs more work, Linux is more like the bastard child, Not a 1:1 code comparison vs. up to UNIX v4

Apple has plans to quit x86_64 in 2020 and use only ARM64 CPU`s the fastest 64-core/128-threads overclocked AMD dual Epyc cpus in the world, cannot run windows in realtime priority, OSX has time bombs, programmed obsolecense, but Windows&AMD are also doing the samething with Win8.1, thats the reason i moved to OSX, Not hackintosh, Real Apple HW,

Apple EFI is very different, "Not GUI Bios like Windows Boards" sometimes NVRam & SMC gets corrupted, its a bit tricky to get it working again. Hyperthreading must be dissabled with a software in OSX, Not in Bios like Windows. OSX has free RamDrive, Windows has AMD / DataRam RAMDisk limited to 4GB free version. but memory used as a HDD/SDD seems faster in OSX, doing speed tests with BlackMagic Disk Speed Test vs HDTach same ddr3 1033 1333, OSX has ECC U / E, Not R, Windows gamer boards do Not have ECC in Desktops. Quality vs. Quantity. Apple is very limited in HW, but those boundries gives a sense of stability. Windows10 is also falling in the drivers incompatibility-

simply is the 2nd time i get bored with Windows. Linux was too limited in SW, has problems with backward compatibility, similar to OSX but worse. OSX is too limited in HW, but SW library is less limited than Linux. Nothing is perfect, but for Now i like OSX GUI more,

OSX is like an improved Linux with as much SW library as Windows, Not as limited as Linux, but bonded/married with x or y hardware. Linux is the opposite, More HW but less SW, i think its pointless in some situations.

Silicon Graphics SGI Iris / Indigo, etc... also had similar fluid graphics as OSX, AmigaONE x5000 PowerPC version of old Motorola AMIGA 500/1000/2000/4000 looks interesting, but... is limited in both HW & SW.

anyway... this thread remembers me this: http://bardagjy.com/?p=1372 & http://blog.nwoolls.com/2013/04/24/bitcoin-mining-on-mac-os-x-cgminer-bfgminer/

juanpc2018 commented 5 years ago

installing Linux in OSX with Bootcamp is possible... https://en.wikipedia.org/wiki/Boot_Camp_(software)#Limitations

but the only advantage of installing Windows would be to use Macs Fan Control, very important... https://www.crystalidea.com/macs-fan-control/download