NTAG215 old MFU format conversion not working?

shellster commented 4 years ago

Describe the bug If I use the very old (deprecated) iceman fork found here: https://github.com/iceman1001/proxmark3, I can use a tool like https://raw.githubusercontent.com/tomvanveen/samytools/master/mfubin2eml to clone an amiibo to a Proxmark3 Easy and successfully use it with a Nintendo switch. However, if I use the latest version (proxmark3-4.9237) from this repo (and same amiibo file that works with the deprecated version) I get the following warnings:

proxmark3 /dev/ttyACM0 -c 'hf mf eload u amiibo'
[=] Session log <redacted>/log_20200705.txt
[=] Loading Preferences...
[+] loaded from JSON file <redacted>/preferences.json
[+] execute command from commandline: hf mf eload u amiibo

[=] Using UART port /dev/ttyACM0
[=] Communicating with PM3 over USB-CDC
[usb|script] pm3 --> hf mf eload u amiibo
[+] loaded 1020 bytes from text file amiibo.eml
[+] old mfu dump format, was converted on load to 243 pages
[=] Uploading to emulator memory
.................................................................................................................................................................................................................................................................

[-] ⛔ Warning, Ultralight/Ntag file content, Loaded 257 blocks into emulator memory

proxmark3 /dev/ttyACM0 -c 'hf 14a sim t 7 u 0418521a454b81'
[=] Session log <redacted>/log_20200705.txt
[=] Loading Preferences...
[+] loaded from JSON file <redacted>/preferences.json
[+] execute command from commandline: hf 14a sim t 7 u 0418521a454b81

[=] Using UART port /dev/ttyACM0
[=] Communicating with PM3 over USB-CDC
[usb|script] pm3 --> hf 14a sim t 7 u 0418521a454b81
[+] Emulating ISO/IEC 14443 type A tag with 7 byte UID (04 18 52 1A 45 4B 81 )
[=] Press pm3-button to abort simulation

When I try to use the amiibo the Nintendo switch does not recognize it as a valid amiibo. I recognize that the tag file format has changed since the deprecated version (https://github.com/RfidResearchGroup/proxmark3/blob/8aebb8ef8ebf1f6301d8992c9054c87995cf4a01/doc/mfu_binary_format_notes.md), but I think there is some sort of bug in how the old format is being converted, since it is not resulting in a valid amiibo. I'm happy to provide logs, or specific files if that is helpful. If someone can point me to some documentation that can help me update my .eml files to work with the new version, I'd be grateful.

To Reproduce Steps to reproduce the behavior:

Clone https://github.com/iceman1001/proxmark3
cd into the directory.
make
cd client
./flasher -b ../bootrom/obj/bootrom.elf ../armsrc/obj/fullimage.elf
Unplug/replug proxmark3.
(Referencing amiibo.eml that I can provide, in this case for Animal Crossing Lily) copy amiibo.eml to current folder.

./proxmark3 /dev/ttyACM0 -c 'hf mf eload u amiibo'
./proxmark3 /dev/ttyACM0 -c 'hf 14a sim t 7 u 0430a2f28c4b80'

Test cloned amiibo and validate that it works.
Download latest relase of iceman proxmark repo.
untar.
cd into release folder.
make PLATFORM=PM3OTHER (I'm testing with a Proxmark3 Easy, so need PM3OTHER)
./pm3-flash-all
cd client
copy amiibo.eml to current folder.

./proxmark3 /dev/ttyACM0 -c 'hf mf eload u amiibo'
./proxmark3 /dev/ttyACM0 -c 'hf 14a sim t 7 u 0430a2f28c4b80'

Observe that this time the amiibo is not valid.

Expected behavior I expect that I should be able to make a successful amiibo clone using the same amiibo.eml file and commands as I can on the deprecated code base.

Screenshots N/A

Desktop (please complete the following information):

OS: Debian Stretch (also validated on Raspbian)
inside proxmark3 client run the following commands and paste the output here.
hw version
hw status
data tune

[usb] pm3 --> hw version

 [ Proxmark3 RFID instrument ]

 [ CLIENT ]
  client: RRG/Iceman/master/release (git) 
  compiled with GCC 6.3.0 20170516 OS:Linux ARCH:x86_64

 [ PROXMARK3 ]

 [ ARM ]
  bootrom: RRG/Iceman/master/release (git) 
       os: RRG/Iceman/master/release (git) 
  compiled with GCC 5.4.1 20160919

 [ FPGA ]
  LF image built for 2s30vq100 on 2020-02-22 at 12:51:14
  HF image built for 2s30vq100 on 2020-01-12 at 15:31:16

 [ Hardware ]
  --= uC: AT91SAM7S256 Rev C
  --= Embedded Processor: ARM7TDMI
  --= Nonvolatile Program Memory Size: 256K bytes, Used: 261435 bytes (100%) Free: 709 bytes ( 0%)
  --= Second Nonvolatile Program Memory Size: None
  --= Internal SRAM Size: 64K bytes
  --= Architecture Identifier: AT91SAM7Sxx Series
  --= Nonvolatile Program Memory Type: Embedded Flash Memory

[usb] pm3 --> hw status
#db# Memory
#db#   BIGBUF_SIZE.............40000
#db#   Available memory........40000
#db# Tracing
#db#   tracing ................1
#db#   traceLen ...............0
#db# Currently loaded FPGA image
#db#   mode.................... HF image built for 2s30vq100 on 2020-01-12 at 15:31:16
#db# LF Sampling config
#db#   [q] divisor.............95 ( 125.00 kHz)
#db#   [b] bits per sample.....8
#db#   [d] decimation..........1
#db#   [a] averaging...........Yes
#db#   [t] trigger threshold...0
#db#   [s] samples to skip.....0 
#db# LF T55XX config
#db#            [r]               [a]   [b]   [c]   [d]   [e]   [f]   [g]
#db#            mode            |start|write|write|write| read|write|write
#db#                            | gap | gap |  0  |  1  | gap |  2  |  3
#db# ---------------------------+-----+-----+-----+-----+-----+-----+------
#db# fixed bit length (default) |  31 |  20 |  18 |  50 |  15 | N/A | N/A | 
#db#     long leading reference |  31 |  20 |  18 |  50 |  15 | N/A | N/A | 
#db#               leading zero |  31 |  20 |  18 |  40 |  15 | N/A | N/A | 
#db#    1 of 4 coding reference |  31 |  20 |  18 |  34 |  15 |  50 |  66 | 
#db# 
#db# Transfer Speed
#db#   Sending packets to client...
#db#   Time elapsed............500ms
#db#   Bytes transferred.......305664
#db#   Transfer Speed PM3 -> Client = 611328 bytes/s
#db# Various
#db#   DBGLEVEL................1
#db#   ToSendMax...............-1
#db#   ToSendBit...............0
#db#   ToSend BUFFERSIZE.......2308
#db#   Slow clock..............30570 Hz
#db# Installed StandAlone Mode
#db#   HF - Reading Visa cards & Emulating a Visa MSD Transaction(ISO14443) - (Salvador Mendoza)
[usb] pm3 --> data tune
[=] Measuring antenna characteristics, please wait...
 🕛   9
[+] LF optimal: 542862.64 V -   0.00 kHz
[+] LF antenna is OK 

[+] HF antenna: 25.52 V - 13.56 MHz
[+] HF antenna is OK 

[-] ⛔ Not showing LF tuning graph since all values is zero.

LF Tuning is because I currently have removed the LF antenna.

Additional context N/A

shellster commented 4 years ago

Accidentally submitted this before I was done typing up the bug issue. So I closed the issue until I was done editing. Reopening now.

iceman1001 commented 4 years ago

I pushed some minor changes, one is the addition of a amiboo pack when used with a dumpfile which didn't have it. Pull latest and see if it works better.

Side note: Your Proxmark3 device has 256kb internal flash memory and the full image you flashed is really pushing the limits. Try to remove some command support from the armsrc makefile, to generate a smaller fullimage.

  --= Nonvolatile Program Memory Size: 256K bytes, Used: 261435 bytes (100%) Free: 709 bytes ( 0%)

shellster commented 4 years ago

I'll check it next week when I get home and have another computer. Thanks for the response. I tried building the latest from master, but I'm running into the same errors as described in https://github.com/RfidResearchGroup/proxmark3/issues/815 I'm on Devuan Linux. I probably have an Ubuntu VM somewhere, at home.

mokus0 commented 4 years ago

I've seen this same issue on my system (rdv4, tested on multiple OSes) - the same .eml file works on an old build, but not on new one, including master as of today (commit 551bc6c71bf41d6d8b7f3a876ded720a81cbca73). Happy to provide any info needed.

mokus0 commented 4 years ago

Looking at the header formats and poking around in the simulator memory between the two versions, I notice a couple things:

In the header as loaded by the current master, the "version" field appears to be truncated to the lower 4 bytes (from the eml file, it should be 0004040201001103, but the first 8 bytes of hf mf eget 0 are 0000000001001103.
Second, I'm not sure if it's actually wrong or not, but the pages field is loaded as 0xF2, whereas in the old version it prints a message after eload saying that it loaded 255 blocks.

I've tried manually fixing the version using hf mf eset, with and without also tweaking the pages field, but that wasn't enough to actually make it work.

Syfaro commented 4 years ago

@cheeplusplus wrote about they got amiibo support working again here: https://blog.neocodenetworks.com/posts/2020-04-08-amiibo-emulation/

https://github.com/cheeplusplus/proxmark3/commit/52184833e5c0d6066db6e5009b48dd3c00629e01

mokus0 commented 4 years ago

Thanks, looks like the password part is probably the other piece I was missing. The password value doesn't end up in the simulator memory with the new legacy load operation. If my quick skim of the code is correct, it appears to be pulling it from address 1024. I need to get to bed so I didn't dig any deeper to see whether that's a valid place to load from, but I can't access that via hf mf eget anyway.

shellster commented 4 years ago

Thanks guys for all your testing as well as the previous link.

mokus0 commented 4 years ago

I believe I've tracked down the cause of the 0's in the first 4 bytes loaded. They are actually the last 4 bytes - in the generated eml files from samyk's mfubin2eml tool, there are 1020 bytes including the old mfu header. when read in and converted to the new format, the resulting data is now 1028 bytes due to the larger header. The extra 4 bytes beyond 1024 overflow and end up written to bytes 0-3. Manually deleting the last block from the eml file fixes the 'version' field.

mokus0 commented 4 years ago

Ok, and I've now figured out a bit more about the password part. The Switch sends an auth request with some kind of a password to the card, and the firmware is pulling the expected password out of the second-to-last block of the dump. If it matches the value sent by the switch, It then sends a response from the last block in the dump. So with some adjustments to the eml files only, I'm able to get it working on the current master.

To summarize, I had to:

shorten the eml file
append the expected challenge code, which is different for each amiibo (I got it for a couple amiibos by just adding some debug printing to the firmware, but I'm guessing there's a way to calculate this) - it'd also be nice to have a firmware option to ignore it, like @cheeplusplus does in their branch.
append the response code that the amiibo is supposed to send back (80808080)

It would also be good to change the header generated by mfubin2eml to the new format. All of these changes should be feasible in mfubin2eml, assuming the challenge code can be calculated.

I'm beat for the night, and I don't know when I might have time to poke at this more, but that should at least be a good amount of info if someone else wants to run with it.

cheeplusplus commented 4 years ago

@mokus0 If you're talking about calculating the PWD to check if you should send pACK, the code for that is already in proxmark:

https://github.com/RfidResearchGroup/proxmark3/blob/f830843442664eb70ad0361a4d7eee8e30044021/common/generator.c#L99-L109

If I recall correctly, part of the reason I hardcoded it was because I was using unencrypted amiibo dumps with pm3_amii_bin2eml.pl, which (if not in the dump) adds a header using a fixed UID and 00000000 for the PWD, and therefore a non-special-cased MIFARE_ULEV1_AUTH command would fail. You would have to either use a properly encrypted dump (DIY dump or re-encrypt with amiitool), or alternatively modify the bin2eml script to calculate the real PWD and put it in the header.

mokus0 commented 4 years ago

Yeah, I suspect that's exactly what I'd need - thanks! My plan if I have the time in the next day or so is to update the bin2eml script to generate the new header format, including only as much data as was in the original binary dump, and add the extra auth tokens. Your hardcoded fix of ignoring the PWD works well in the interim.

kprocbert commented 3 years ago

can confirm that it's working with the new mfu conversion script.

One question tho, when i try to write one of my chinese ntags with:

old version:

hf mf cload myfile
[+] loaded 1020 bytes from text file myfile.eml
[!!] 🚨 File content error.

new version:

hf mf cload myfile
[+] loaded 596 bytes from text file myfile.eml
[!!] 🚨 File content error.

any ideas? :/

iceman1001 commented 3 years ago

@mokus0 I think we have the same script from samy, its just renamed, so it would be nice if you did the same love to it as you did the upstream one :)

https://github.com/RfidResearchGroup/proxmark3/blob/master/tools/pm3_amii_bin2eml.pl

mokus0 commented 3 years ago

@iceman1001 sure, I can do that - it might be a couple days before I have the spare cycles though!

fbettag commented 3 years ago

can confirm that it's working with the new mfu conversion script.

One question tho, when i try to write one of my chinese ntags with:

old version:
hf mf cload myfile
[+] loaded 1020 bytes from text file myfile.eml
[!!] 🚨 File content error. 
new version:
hf mf cload myfile
[+] loaded 596 bytes from text file myfile.eml
[!!] 🚨 File content error. 
any ideas? :/

Same issue here

iceman1001 commented 3 years ago

until you compare the files there is nothing to be said.

iceman1001 commented 2 years ago

closing because of inactivity.

if someone comes along and test different old style mfu binaries and can verify it still doesn't work we can reopen this one and take it from there

RfidResearchGroup / proxmark3

NTAG215 old MFU format conversion not working? #832