Static macOS build releases

[floyd-fuh]

thank you for the tool, is there any chance that there will be a MacOS static compile release of this? boost is currently breaking the build for me and I guess it takes a while to figure it out... and it seems that the build.sh works for you on MacOS

[cr-marcstevens]

Hi Floyd, Unfortunately, I can't as I don't have access to any Mac. The additional code in build.sh for Mac comes from another Mac user.

If you happen to figure it out, could you send me some brief hints for other Mac users?

[floyd-fuh]

Ok, figured it out, don't cancel the install_boost.sh, otherwise it will detect in the next run the boost folder that was created, which of course is corrupt. Anyway it would be nice to have static compiles :)

[floyd-fuh]

Oh and in scripts/poc_no.sh usually on mac, binary md5sum is only called md5 and binary sha1sum is called shasum (and does SHA-1 by default). Otherwise you get:

./scripts/poc_no.sh: line 144: md5sum: command not found (also at some more places!) ./scripts/poc_no.sh: line 145: sha1sum: command not found

If all calls are changed to md5 and shasum it works fine. Otherwise it will run in an endless loop as it thinks that the md5 are not the same and therefore no collision was found.

[floyd-fuh]

cpc.sh doesn't work on Mac, because it is using /proc/. If you hard code the CPU amount, it will pass that part and then fail with:

$ ./cpc.sh prefix1 prefix2
Chosen-prefix file 1: prefix1
Chosen-prefix file 2: prefix2
Detecting worklevel..../cpc.sh: command substitution: line 45: syntax error near unexpected token `&'
./cpc.sh: command substitution: line 45: `$BIRTHDAYSEARCH --inputfile1 "$file1" --inputfile2 "$file2" --hybridbits 0 --pathtyperange 2 --maxblocks 9 --maxmemory 100 --threads $CPUS --cuda_enable |& grep "^Work" -m1 | head -n1 | cut -d'(' -f2 | cut -d'.' -f1'
: 
./cpc.sh: line 47: [: -ge: unary operator expected

[cr-marcstevens]

Hi Floyd,

Thanks for all the info, I can try to see if we can make the scripts a bit Mac-wise. Regarding cpc.sh, can you try to replace all pipe cout&cerr |& with just a regular pipe |, and see if that works?

[floyd-fuh]

Hi Marc,

Thanks for the quick replies. Yes, it seems to run with | instead of |&. Unfortunately, I don't have time to do a full run right now, but it seems to be computing fine.

fyi, there's a warning about OS randomness but the rest looks good to me (read: to me as a complete beginner who hasn't even started to read the code properly):

$ ./cpc.sh prefix1 prefix2
Chosen-prefix file 1: prefix1
Chosen-prefix file 2: prefix2
Detecting worklevel...: 30
Warning: no OS randomness!
Birthday search for MD5 chosen-prefix collisions
Copyright (C) 2009 Marc Stevens
http://homepages.cwi.nl/~stevens/

IHV1 = {1732584193,4023233417,2562383102,271733878}
IHV1 = 0123456789abcdeffedcba9876543210

IHV2 = {1732584193,4023233417,2562383102,271733878}
IHV2 = 0123456789abcdeffedcba9876543210

Maximum of near-collision blocks: 9
Differential Path Type range: 2
Hybrid bits: 0
Maximum amount of memory in MB for trails: 100 (local: 100)
Estimated number of trails that will be stored:    1872457 (local: 1872457)
Estimated number of trails that will be generated: 1061342
Estimated complexity per trail: 2^(17)
Estimated complexity on trails: 2^(37.0175)
Estimated complexity on collisions: 2^(27.7053)

Thread 1 created (AVX256).
Thread 2 created (AVX256).
Work: 2^(30.0328), Coll.: 0(uf=0,nuf=0,?=0,q=0,rh=0), Blocks: 64
Work: 2^(31.0092), Coll.: 0(uf=0,nuf=0,?=0,q=0,rh=0), Blocks: 64
Work: 2^(31.576), Coll.: 0(uf=0,nuf=0,?=0,q=0,rh=0), Blocks: 64
Work: 2^(31.9796), Coll.: 0(uf=0,nuf=0,?=0,q=0,rh=0), Blocks: 64
Work: 2^(32.2589), Coll.: 1(uf=0,nuf=1,?=0,q=0,rh=0), Blocks: 64
Work: 2^(32.5205), Coll.: 2(uf=0,nuf=2,?=0,q=0,rh=0), Blocks: 64
Work: 2^(32.748), Coll.: 2(uf=0,nuf=2,?=0,q=0,rh=0), Blocks: 64
Work: 2^(32.9507), Coll.: 2(uf=0,nuf=2,?=0,q=0,rh=0), Blocks: 64

Work: 2^(33.0666), Coll.: 3(uf=1,nuf=2,?=0,q=0,rh=0), Blocks: 14
Work: 2^(33.1213), Coll.: 4(uf=1,nuf=3,?=0,q=0,rh=0), Blocks: 14
Work: 2^(33.2685), Coll.: 4(uf=1,nuf=3,?=0,q=0,rh=0), Blocks: 14
Work: 2^(33.3982), Coll.: 5(uf=1,nuf=4,?=0,q=0,rh=0), Blocks: 14
Work: 2^(33.5151), Coll.: 5(uf=1,nuf=4,?=0,q=0,rh=0), Blocks: 14
Work: 2^(33.6283), Coll.: 5(uf=1,nuf=4,?=0,q=0,rh=0), Blocks: 14
Work: 2^(33.7336), Coll.: 5(uf=1,nuf=4,?=0,q=0,rh=0), Blocks: 14
Work: 2^(33.8278), Coll.: 5(uf=1,nuf=4,?=0,q=0,rh=0), Blocks: 14
Work: 2^(33.925), Coll.: 5(uf=1,nuf=4,?=0,q=0,rh=0), Blocks: 14
Work: 2^(34.0162), Coll.: 5(uf=1,nuf=4,?=0,q=0,rh=0), Blocks: 14
Work: 2^(34.0962), Coll.: 8(uf=1,nuf=7,?=0,q=0,rh=0), Blocks: 14
Work: 2^(34.1758), Coll.: 8(uf=1,nuf=7,?=0,q=0,rh=0), Blocks: 14

Work: 2^(34.1804), Coll.: 9(uf=2,nuf=7,?=0,q=0,rh=0), Blocks: 12
Work: 2^(34.2529), Coll.: 9(uf=2,nuf=7,?=0,q=0,rh=0), Blocks: 12
Work: 2^(34.3277), Coll.: 11(uf=3,nuf=8,?=0,q=0,rh=0), Blocks: 12
Work: 2^(34.3972), Coll.: 13(uf=3,nuf=10,?=0,q=0,rh=0), Blocks: 12
Work: 2^(34.4511), Coll.: 15(uf=4,nuf=11,?=0,q=0,rh=0), Blocks: 12
Work: 2^(34.5145), Coll.: 16(uf=4,nuf=12,?=0,q=0,rh=0), Blocks: 12

Work: 2^(34.5217), Coll.: 17(uf=5,nuf=12,?=0,q=0,rh=0), Blocks: 11
Work: 2^(34.5696), Coll.: 18(uf=5,nuf=13,?=0,q=0,rh=0), Blocks: 11
Work: 2^(34.6191), Coll.: 20(uf=6,nuf=14,?=0,q=0,rh=0), Blocks: 11
Work: 2^(34.6709), Coll.: 22(uf=8,nuf=14,?=0,q=0,rh=0), Blocks: 11
Work: 2^(34.7276), Coll.: 24(uf=10,nuf=14,?=0,q=0,rh=0), Blocks: 11
Work: 2^(34.7823), Coll.: 25(uf=10,nuf=15,?=0,q=0,rh=0), Blocks: 11
Work: 2^(34.8344), Coll.: 26(uf=11,nuf=15,?=0,q=0,rh=0), Blocks: 11
Work: 2^(34.8846), Coll.: 28(uf=12,nuf=16,?=0,q=0,rh=0), Blocks: 11
Work: 2^(34.9314), Coll.: 29(uf=12,nuf=17,?=0,q=0,rh=0), Blocks: 11

Work: 2^(34.9536), Coll.: 30(uf=13,nuf=17,?=0,q=0,rh=0), Blocks: 10
Work: 2^(34.9774), Coll.: 32(uf=15,nuf=17,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.0221), Coll.: 32(uf=15,nuf=17,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.0593), Coll.: 33(uf=16,nuf=17,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.0978), Coll.: 34(uf=16,nuf=18,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.1327), Coll.: 34(uf=16,nuf=18,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.1718), Coll.: 34(uf=16,nuf=18,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.2089), Coll.: 36(uf=17,nuf=19,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.245), Coll.: 37(uf=18,nuf=19,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.2807), Coll.: 38(uf=19,nuf=19,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.3177), Coll.: 39(uf=19,nuf=20,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.3503), Coll.: 40(uf=19,nuf=21,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.382), Coll.: 43(uf=20,nuf=23,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.4151), Coll.: 44(uf=21,nuf=23,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.445), Coll.: 46(uf=22,nuf=24,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.4754), Coll.: 47(uf=23,nuf=24,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.5019), Coll.: 50(uf=24,nuf=26,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.5293), Coll.: 52(uf=26,nuf=26,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.556), Coll.: 55(uf=28,nuf=27,?=0,q=0,rh=0), Blocks: 10
Work: 2^(35.5829), Coll.: 59(uf=30,nuf=29,?=0,q=0,rh=0), Blocks: 10

Thank you very much for the tools, I definitely got a better understanding of how md5 collisions can be exploited due to your work here and ange's workshop. While I've done a quick fastcol as a PoC before to demonstrate/analyze systems that use md5, I'm now much more confident to be able to create better PoCs for systems using md5.

[noloader]

@cr-marcstevens,

Sorry to poke my nose in here...

I can provide remote SSH access to a MacBook Core2 running OS X 10.9 with Xcode 7 and a Mac Mini Core-i5 running OS X 10.12 with cctools (no Xcode). I also have a old PowerMac G5 running OS X 10.5 with Xcode 3, but you probably are not interested in it.

Emal me your SSH public key or authorized_keys, if interested. noloader, gmail account.

[evandrix]

bump

[cr-marcstevens]

@evandrix : I have zero experience with MacOS, but I'm open to anyone coming with suggestions how to improve compatibility with MacOS and/or provide (static) build releases in an automatic and lasting manner?

[cr-marcstevens]

Final note: I don't yet see how I could provide static binaries for macos in a systematic and prolonged manner. However, today I added Github actions and CI on both ubuntu-latest and macos-latest.

[gregsadetsky]

Just a word of thanks to everyone for the notes on how to run this on macOS! It ran pretty much out of the box after these changes -- this is on a M1 (i.e. arm64) Mac running macOS 12.4.

A summary of the changes noted above:

• in poc_no.sh, replace md5sum with md5 and sha1sum with shasum
• in cpc.sh, hardcode the CPUS= value
• in cpc.sh again, replace all |& with |

poc_no.sh prefix.txt correctly generates collision1.bin and collision2.bin after that. Cheers

[varenc]

thanks @gregsadetsky ! I was able to get cpc.sh working on macOS by making the changes you outlined. Thanks!

[oregonpillow]

Just a word of thanks to everyone for the notes on how to run this on macOS! It ran pretty much out of the box after these changes -- this is on a M1 (i.e. arm64) Mac running macOS 12.4.

A summary of the changes noted above:

• in poc_no.sh, replace md5sum with md5 and sha1sum with shasum
• in cpc.sh, hardcode the CPUS= value
• in cpc.sh again, replace all |& with |

poc_no.sh prefix.txt correctly generates collision1.bin and collision2.bin after that. Cheers

also worked for me on M3 Pro (14.3.1 ) Had to also run brew install autoconf automake libtool