valgrind / gdb cmdhfmfhard.c

[iceman1001]

With @matrix latest changes, I started to get some crashes. Thanks for the tip on using valgrind. Out of the box it doesn't work because the pm3 client is compiled with -o3. Change that to -o1 and you can use valgrind better. Hook it up with GDB, and when running the 'hf mf hardnested' command it now breaks as seen below. My first guess is that the comment made by @piwi,
"// use logarithms to avoid overflow with huge factorials (double type can only hold 170!)"

Program received signal SIGILL, Illegal instruction.
[Switching to Thread 5740]
p_hypergeometric (N=N@entry=256, K=K@entry=32, n=n@entry=4, k=k@entry=0) at cmdhfmfhard.c:264
264         return exp(log_result);
(gdb) 

[iceman1001]

#0  p_hypergeometric (N=N@entry=256, K=K@entry=32, n=n@entry=10, k=k@entry=0) at cmdhfmfhard.c:264
#1  0x0808260b in p_hypergeometric (N=N@entry=256, K=K@entry=32, n=n@entry=10, k=k@entry=1) at cmdhfmfhard.c:276
#2  0x0808260b in p_hypergeometric (N=N@entry=256, K=K@entry=32, n=n@entry=10, k=k@entry=2) at cmdhfmfhard.c:276
#3  0x0808260b in p_hypergeometric (N=N@entry=256, K=K@entry=32, n=n@entry=10, k=k@entry=3) at cmdhfmfhard.c:276
#4  0x0808260b in p_hypergeometric (N=N@entry=256, K=<optimized out>, n=n@entry=10, k=k@entry=4) at cmdhfmfhard.c:276
#5  0x080826ec in sum_probability (k=4, n=10, K=32) at cmdhfmfhard.c:287
#6  estimate_second_byte_sum () at cmdhfmfhard.c:577
#7  0x08082e3a in acquire_nonces (blockNo=blockNo@entry=0 '\000', keyType=keyType@entry=0 '\000', 
    key=key@entry=0x6291af0 <incomplete sequence \374>, trgBlockNo=trgBlockNo@entry=36 '$', 
    trgKeyType=trgKeyType@entry=0 '\000', nonce_file_write=nonce_file_write@entry=false, slow=slow@entry=false)
    at cmdhfmfhard.c:843
#8  0x0808587a in mfnestedhard (blockNo=blockNo@entry=0 '\000', keyType=keyType@entry=0 '\000', 
    key=key@entry=0x6291af0 <incomplete sequence \374>, trgBlockNo=trgBlockNo@entry=36 '$', 
    trgKeyType=trgKeyType@entry=0 '\000', trgkey=trgkey@entry=0x0, nonce_file_read=false, 
    nonce_file_write=nonce_file_write@entry=false, slow=slow@entry=false, tests=tests@entry=0) at cmdhfmfhard.c:1817
#9  0x08078dba in CmdHF14AMfNestedHard (Cmd=<optimized out>) at cmdhfmf.c:1062
#10 0x08090b08 in CmdsParse (Commands=Commands@entry=0x81134e0 <CommandTable>, Cmd=<optimized out>, 
    Cmd@entry=0xbef1166 "hardnest 0 a fc00018778f7 36 a") at cmdparser.c:69
#11 0x0807c100 in CmdHFMF (Cmd=0xbef1166 "hardnest 0 a fc00018778f7 36 a") at cmdhfmf.c:2510
#12 0x08090b08 in CmdsParse (Commands=Commands@entry=0x8112f80 <CommandTable>, Cmd=<optimized out>, 
    Cmd@entry=0xbef1163 "mf hardnest 0 a fc00018778f7 36 a") at cmdparser.c:69
#13 0x08068cee in CmdHF (Cmd=0xbef1163 "mf hardnest 0 a fc00018778f7 36 a") at cmdhf.c:945
#14 0x08090b08 in CmdsParse (Commands=Commands@entry=0x8113fc0 <CommandTable>, Cmd=<optimized out>, 
    Cmd@entry=0xbef1160 "hf mf hardnest 0 a fc00018778f7 36 a") at cmdparser.c:69

[iceman1001]

http://pastebin.com/UCKYUUxJ

[iceman1001]

Almost forgot the varible value. Seems like a negative float and exp doesnt like eachother

log_result = -1,3611116341317357

[matrix]

https://github.com/iceman1001/proxmark3/blob/master/client/cmdhfmfhard.c#L910-L911

Retry using the commented original values

[iceman1001]

Well, I don't know if this nonces.bin file is wrong pre se, but it caused a seg kill when it was made. https://www.sendspace.com/file/jrgi9u

[osysltd]

seems hardnested is broken currently btw, doesn't work, very strange behavior (number of bytes going down) and always the same rollback byte (no matter what sectors\key types)

--target block no: 63, target key type:A, known target key: 0x000000000000 (not set), file action: none, Slow: Yes, Tests: 0
Allocating memory for partial statelists... Generating partial statelists... Generating bitflip statelist... Acquiring nonces... Checking for Filter Flip Properties... Acquired 1568 nonces ( 1548 with distinct bytes 0 and 1). Number of bytes with probability for correctly guessed Sum(a8) > 95.0%: 13 Acquired 2016 nonces ( 1978 with distinct bytes 0 and 1). Number of bytes with probability for correctly guessed Sum(a8) > 95.0%: 9 Acquired 2576 nonces ( 2514 with distinct bytes 0 and 1). Number of bytes with probability for correctly guessed Sum(a8) > 95.0%: 1 Acquired 3024 nonces ( 2942 with distinct bytes 0 and 1). Number of bytes with probability for correctly guessed Sum(a8) > 95.0%: 0 Acquired 3584 nonces ( 3468 with distinct bytes 0 and 1). Number of bytes with probability for correctly guessed Sum(a8) > 95.0%: 0 Acquired 4032 nonces ( 3890 with distinct bytes 0 and 1). Number of bytes with probability for correctly guessed Sum(a8) > 95.0%: 0 Generating crypto1 state candidates... Number of possible keys with Sum(a0) = 128: 117726714265600 (2^46.7) Number of remaining possible keys: 482916425164 (2^38.8) Time for generating key candidates list: 5 seconds
Brute force phase starting.
Using 128-bit bitslices
Bitslicing best_first_byte^uid[3](rollback byte): 5D ...
Bitslicing nonces...
Starting 2 cracking threads to search 34 buckets containing a total of 482916425164 states...

[iceman1001]

Yeah, I noticed the counting down of guessed sum(a8) , funny. Did it find the key?

[osysltd]

Nope, hf mf hard is broken currently.

[iceman1001]

Partly broken, since I and @matrix can get it to solve keys. but I agree not stable at all.

[osysltd]

I've tried for two days against two sectors using different combination of known sectors\keys with no luck ;(

[iceman1001]

if you add "w" to your command, and share the collected nonce file, that might help out while debugging.

[iceman1001]

Looking at generating_candidates call, it eventually ends up in the p_hypergeometric function. With some comments from piwi, and the sigill comes.. @aczid thought it could be a memalignment problem. Some mallac not correct?

[osysltd]

I've uploaded nonces and log here: https://www.idrive.com/idrive/sh/sh?k=d7k8x1v3g6

[osysltd]

d209443322a410eadd9746ac74c6a8b4899788a9 works great with the same conditions

[osysltd]

Per my tests the best outcome is from f6c56cd204e1548b3cf4404b44f44fb3e13b45d9 revision.

[matrix]

Hi @Osys, what's about your tests? I see in your log an unknown bootrom name/version, different that os version. Also your proxmark hardware seems old revision, all correct?

On Wednesday, November 2, 2016, Osys notifications@github.com wrote:

Per my tests the best outcome is from f6c56cd https://github.com/iceman1001/proxmark3/commit/f6c56cd204e1548b3cf4404b44f44fb3e13b45d9 revision.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iceman1001/proxmark3/issues/48#issuecomment-257872951, or mute the thread https://github.com/notifications/unsubscribe-auth/AOqywdpw2tRRhzcFNqz42a082hhynhxTks5q6Jf1gaJpZM4KgakK .

[osysltd]

Hi Matrix, bootrom never flashed, stock one (from china). HW version is Easy. Os version is dirty due to change in common/Makefile (comment out APP_CFLAGS HAS_512_FLASH). No more changes though...

Currently os image from last revision however client side is from this commit f6c56cd204e1548b3cf4404b44f44fb3e13b45d9

[matrix]

I'm not Iceman and I think you need flash also the bootrom before do other "test" :)

Thanks, matrix

On Wednesday, November 2, 2016, Osys notifications@github.com wrote:

Hi Iceman, bootrom never flashed, stock one (from china). HW version is Easy. Os version is dirty due to change in common/Makefile (comment out APP_CFLAGS HAS_512_FLASH). No more changes though...

Currently os image from last revision however client side is from this commit f6c56cd https://github.com/iceman1001/proxmark3/commit/f6c56cd204e1548b3cf4404b44f44fb3e13b45d9

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iceman1001/proxmark3/issues/48#issuecomment-257877463, or mute the thread https://github.com/notifications/unsubscribe-auth/AOqywXdu8UM86rG6wBfrhW6E_CAuCxZ7ks5q6JuFgaJpZM4KgakK .

[osysltd]

Already corrected, sorry ;) Bootrom have nothing to do with OS image, it is there only to actually flash OS image. As an outcome of some discussions with @iceman1001, Easy is like an old hardware version (green one), yes.

[matrix]

I test the original hardnested attack also in old proxmark hardware and I see a lot of errors like segfault, wrong 13bytes keys, unable to found correct keys, etc. With revision 2 of proxmark the attack work better (tested on linux and osx).

On Wednesday, November 2, 2016, Osys notifications@github.com wrote:

Already corrected, sorry ;) Bootrom have nothing to do with OS image, it is there only to actually flash OS image. As an outcome of some discussions with @iceman1001 https://github.com/iceman1001, Easy is like an old hardware version (green one), yes.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iceman1001/proxmark3/issues/48#issuecomment-257879242, or mute the thread https://github.com/notifications/unsubscribe-auth/AOqywWMP7h0zQnViXWYqKgUTLJK4cUQkks5q6JzxgaJpZM4KgakK .

[osysltd]

Do you have an idea how to get it working no matter what hardware?

[iceman1001]

To be honest, I've lots of problems with "older green" pm3 running "hf mf mifare" or "hf mf hardnested". its like older pm3 is too slow. The RDV1 or RDV2 works fine besides this mem corruption. I'm gonna look into changing the p_hypergeometric function.

[matrix]

Old hw produce unstable results. I think it's better answer that in proxmark forum

On Wednesday, November 2, 2016, Osys notifications@github.com wrote:

Do you have an idea how to get it working no matter what hardware?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iceman1001/proxmark3/issues/48#issuecomment-257898861, or mute the thread https://github.com/notifications/unsubscribe-auth/AOqywSWa43nwVFlwWGrD-fKKOclLjaRLks5q6KtsgaJpZM4KgakK .

[osysltd]

Or revert changes to f6c56cd204e1548b3cf4404b44f44fb3e13b45d9 ;) As it seems the most working solution right now.

[matrix]

Or fork iceman repository and do what you want without ask to nobody ? There're infinite solutions around you :)

On Wednesday, November 2, 2016, Osys notifications@github.com wrote:

Or revert changes to f6c56cd https://github.com/iceman1001/proxmark3/commit/f6c56cd204e1548b3cf4404b44f44fb3e13b45d9 ;) As it seems the most working solution right now.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iceman1001/proxmark3/issues/48#issuecomment-257915611, or mute the thread https://github.com/notifications/unsubscribe-auth/AOqywV6_geKnT9qmKR0IqbdPgBi7UwV0ks5q6Le8gaJpZM4KgakK .

[osysltd]

I'm not so well coder (but Im trying to learn :) therefore only I can help with is provide feedback. And the feedback as follows - currently hf mf hard doesn't work well as it used to...

[matrix]

I'm only a coder, I can only write code ... :)

On Wednesday, November 2, 2016, Osys notifications@github.com wrote:

I'm not so well coder (but Im trying to learn :) therefore only I can help with is provide feedback. And the feedback as follows - currently hf mf hard doesn't work well as it used to...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iceman1001/proxmark3/issues/48#issuecomment-257918075, or mute the thread https://github.com/notifications/unsubscribe-auth/AOqywYgmp4RS2IpngTLSf_xqrGPx4UkOks5q6LmagaJpZM4KgakK .

[osysltd]

Then do please optimize the code :)

[matrix]

Hi @iceman1001 I tried your nonces.bin but it is corrupt ...

[osysltd]

Btw @matrix, my github name is not @osys but @osysltd :)

[iceman1001]

How about you two stop the passive agressive posts? So fine, @osysltd use the older commits that works for you and you will be happy. If you provide with more details than just stating that something isnt working for you its not useful to debug.
@matrix sorry, but the code breaks which leaves the file corrupted. How about deleteing a few bytes from the end? I still think its the hyper_geometric function that fucks up. @pwpiwi has other things to do than looking into this. I got the suggestion from @blapost code doesnt have this limitiation. (found here: http://crapto1.netgarage.org/)

[osysltd]

Nothing as you mention, Chris. Sorry if I've said something wrong, dear friends! I just wanted to provide feedback that I thought might be useful for us to collaborate. Sorry if not.

[iceman1001]

Great, @osysltd would you be so kind and save your nonces (add parameter "w") and share it here. Maybe we can replicate the bug that way. A problem right now is not being able to replicate it.

[edit] and now I saw that you did do that earlier. sorry. forget about this suggestion.

[matrix]

Hi @iceman1001, if you tried without my patches and it work please remove my mods, but if you want to keep it search the bug in what I change, not in the p_hypergeometric() because I don't change nothing in this function.

With my hardware (proxmark rdv2 and mifare classic ev1 card) it work, let me know.

uC: AT91SAM7S512 Rev A
Embedded Processor: ARM7TDMI
Nonvolatile Program Memory Size: 512K bytes. Used: 217142 bytes (41). Free: 307146 bytes (59).
Second Nonvolatile Program Memory Size: None
Internal SRAM Size: 64K bytes
Architecture Identifier: AT91SAM7Sxx Series
Nonvolatile Program Memory Type: Embedded Flash Memory

[iceman1001]

The thing is, for me it works when it finds the key in the first "batch". I'm not removing your mods because they are smart and make sence. I do want it to be stable. I hear what you say about yr hardware, I still don't think that is the problem here. Try another card or (sektor/block) . The mem bug shows when its not found in the first batch. (or thats what I see) [edit] yes, you didnt change hyper_Geomentry function I know but that doesnt mean the bug manifests until it come there.

[matrix]

But I write that for speedup the cracking when its not found in the first batch :) With my hardware it work, found ever the key, first or later key is found and cracked. What's your proxmark version?

I'm trying now one of the hardest sector in my card and I crack it in < 5 minutes. Please remove my mods, you can start by restore the original "sizes_even" array values, here https://github.com/iceman1001/proxmark3/blob/master/client/cmdhfmfhard.c#L912-L913 and re-test.

@osysltd I tried also your nonces.bin and I do not found the key inside (cracking failed)

[matrix]

@iceman1001 , try that

diff --git a/client/cmdhfmfhard.c b/client/cmdhfmfhard.c
index a6b8bc5..acee853 100644
--- a/client/cmdhfmfhard.c
+++ b/client/cmdhfmfhard.c
@@ -274,7 +274,7 @@ static double p_hypergeometric(uint16_t N, uint16_t K, uint16_t n, uint16_t k)
                        for (int16_t i = K+1; i <= N; i++) {
                                log_result -= log(i);
                        }
-                       return exp(log_result);
+                       return (log_result > 0) ? exp(log_result) : 0.0;
                } else {                        // recursion
                        return (p_hypergeometric(N, K, n, k-1) * (K-k+1) * (n-k+1) / (k * (N-K-n+k)));
                }

[GreatMichaelLee]

guys, have read all you posted above, looks like, the function of hardnested is broken to me as well from the latest change, it is always generating logging "fail after xxx times" such like that and infinite loop in such a routine. if you need my nonce file, let me know

[matrix]

Hi, could you retry testing from @osysltd patch-4 commit? Remember to update proxmark firmware after restoring old revision.

Thanks

On Thursday, November 3, 2016, 18688994688 notifications@github.com wrote:

guys, have read all you posted above, looks like, the function of hardnested is broken to me as well from the latest change, it is always generating logging "fail after xxx times" such like that and infinite loop in such a routine. if you need my nonce file, let me know

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iceman1001/proxmark3/issues/48#issuecomment-258100100, or mute the thread https://github.com/notifications/unsubscribe-auth/AOqywffSC7SAdsoWmM3mte3_TJpEwRIXks5q6a6LgaJpZM4KgakK .

[GreatMichaelLee]

hi, I didn't see any repository from @osysltd attached to Proxmark3 or there is no osysltd's merge commit on iceman1001 repository network graph? where is this patch 4 commit located?

[matrix]

Ok, so you can use very older commit for exclude all my mods:

git reset --hard 67cd89033c36aade765946cd9f607e787fec7074

On Thursday, November 3, 2016, Lasersword notifications@github.com wrote:

hi, I didn't see any repository from @osysltd https://github.com/osysltd attached to Proxmark3 or there is no osysltd's merge commit on iceman1001 repository network graph? where is this patch 4 commit located?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/iceman1001/proxmark3/issues/48#issuecomment-258170290, or mute the thread https://github.com/notifications/unsubscribe-auth/AOqywVBOf4xbfEZyK-Jw7zFGfdtEMTvuks5q6fjvgaJpZM4KgakK .

[GreatMichaelLee]

thanks, I will try that and let you know if this working fine .

[matrix]

https://github.com/matrix/proxmark3/commit/869a03c2c6267db16cd1418b9e5f2b9049b9a015

[iceman1001]

I've pushed @matrix latest fix.
This how it look for me: http://pastebin.com/FJMqm2zB

[matrix]

strange output :)

Key Found after testing -2311633482 (2^64.0) out of 13485081692 (2^33.7) keys.
Success! Found 1 keys after 0 seconds

[iceman1001]

indeed strange output. Running it several times, it finds some keys. its like when it starts collecting next 2000 nonces it doesnt recalc the probabilites correct.. and could be the cause for the strange output. Pure guessing from my side.

[iceman1001]

There could also be some variables like "count", "maximum_states" that gets wrong data after first try.

[GreatMichaelLee]

looks still problem, I tried a token which once could be hardnested in this commit, however, nothing logging printed and ten minutes after still nothing there and stuck there. the light is firstly showed on with ACD and C is blinking, however, in 1-2 minutes later, the C is off and AD stick to ON always

[GreatMichaelLee]

I just reset to 67cd890, totally different behavior.... light C will be blinking till found the proper data to start 4 thread to do brute force, however, in the 8690a3c the light C is immediately OFF and stuck there, looks like ,the routine is running into a error dead loop and cannot jump out with heavy CPU load. er....just my guess from the mingw behavior... @matrix could you double check what the possible reason? if necessary I could send a nonce file with 67cd890 for my this token. but not sure it help your debugging or not.

[matrix]

@18688994688 you try individually all the commits after 67cd890* for see if you found the key? I'm not the same hw as you, @iceman1001 you can do?