Re-reverse-engineering the device database

[DavidGriffith]

Created by: chenz

The knowledge how the device database came to be seem to be lost in the ether. It was clearly extracted from the Windows software somehow.

I have here an experimental branch in which I try to recreate that process and eventually be able provide new and updated device descriptions.

A Windows program (dump-infoic-dll-exe) was implemented (on Linux, using MinGW + Wine) that dumps the device database from the original device database (InfoIC.dll) in JSON format. Currently only the dll from version 6.60 is supported, but it should be easy to add other versions. 6.60 is already far ahead of what is currently in devices.h (see below).

Functionality was added to minipro-query-db to dump the database, also in JSON format (this is a "flat" version without vendor information).

A Python script (check_db.py) was written to compare the database dumps. It currently reports 3145 new parts and 390 changes values.

There are still a couple of bytes unaccounted for:

• threre are two fields in the original database that don't correspond directly to anything in device_t
• there is one field in device_t (write_unlock) that does not correspond directly to anything in the windows database

Interestingly, there seems to be no additional information regarding AVR fuses in the original database. I looked at several devices with different fuse configurations, but the only fields that differ between them are fields that are already exactly the same in devices.h. So that means either the parameters are hidden in the (yet) undocumented magic numbers of the database, or that information is encoded somewhere else in the original windows software.

[DavidGriffith]

When I emailed @vdudouyt about how he came up with the contents of devices.h, he replied that he reverse-engineered the original tool's code. He has some scripts that he used in this, but he's unable to do anything for a few days on account of other stuff preempting this.

[DavidGriffith]

@vdudouyt got back to me with this information:

Hello,

Sorry, but it looks that I couldn't find anything useful. I'm afraid that in order to fix that problem, we have nothing as to dive into reverse engineering again.

So far here is what I have to tell you if you decide to dive into debugging an original tool:

1. Most of info is stored in InfoIC.dll. That's exactly from where devices.h is coming from.
2. Most of features peculiar to MCUs require special handling. That's really a hard point because most likely we'll need physical samples for each MCU family in order to decently handle that.
3. Placing a breakpoint on "DeviceIoControl" function is a nice idea.

Also, here is a script that was used in order to create devices.h: https://gist.github.com/vdudouyt/822ec164c960c841f7b8

[DavidGriffith]

Created by: chenz

Interesting. The script is incomplete, but I think it dumps the information from a memory image of MiniPro.exe rather than the DLL alone. According to the script, "opts3" contains an index into a table (in MiniPro.exe) that in turn contains the write_unlock information.

[DavidGriffith]

Created by: elosha

Awesome work, we're getting a new chance to look at the Windows IC database. 😸

Will #99 be solved?

I've done some write_unlock research too, without guarantee for completeness:

Value	Unlock procedure used for
0x00	None
0x01	24xx series I2C ROM, 29xx series paged EEPROM, ATtiny, TTL logic test
0x02	RAM test, CMOS logic test, ATtiny (currently ATtiny13x only)
0x03	28xx series parallel EEPROM/Flash, ATtiny, ATmega, AT89 series
0x05	27xx series parallel EPROM, 8751 based MCU
0x34	28xx series parallel EEPROM, 25xx series SPI ROM
0x3e	27xx series parallel EPROM, 28xx series parallel EEPROM
0x1FF	PCF85xx series I2C EPROM
various	PIC12 series MCU
various	PIC16 series MCU
various	PIC18 series MCU

write_unlock even gets used for devices with surely no unlock procedure, like RAMs or logic devices. I wonder if the ICs are getting unlocked … or rather something else, like TL866 pins.

PICxx write_unlock values range from 0x01 to 0x1ff with no apparent pattern. There furthermore seem to be flags in it, like 0x100, and the area 0x0E0 seems to be exclusively used by PICs.

Currently I'm researching protocol_id, which has quite an obvious pattern in it. It really is an identifier for standard operating procedures. Where should I put the results? MD file? HTML file? Github Wiki?

[DavidGriffith]

Created by: chenz

I'm now poking around in Minipro.exe - I'm pretty sure that is what @vdudouyt meant, because address 0x495528, where the write_unlock information was gathered from is not in the memoy map of the dll (but is in the memory map of the exe). There is still lots of stuff in the dll (loads of strings related to fuses for example) that does not seem to be referenced at all though...

[DavidGriffith]

I asked him about what was up with the script. Here is his response:

It's an IDAPython script which is supposed to be ran in the following way:

1. Go to IDA Pro main menu and select "Run Python script"
2. Open InfoIC.dll
3. Run dump_as_c_header() from the Python console.

Personally, I was using Windows edition of IDA Pro in VirtualBox. I guess that it should be run on Linux edition (with IDAPython support) as well after replacing the paths.

[DavidGriffith]

Created by: chenz

Have you checked the program in my fork? It already reproduces all the information in device_t except for write_unlock - which as I said I don't think is in the DLL. I mentioned before that you cannot dump the information with a script, because the tables are partially populated at runtime - so you need to use a Windows debugger (like Valentin), or just load the DLL and use the API (like me).

There is a lot of data in the DLL that does not seem to be referenced by the DLL's code. Is also does not seem to be exported via the symbol table, so I'm not sure if it can be accessed at all. There are still two functions I have not fully understood (GetIcList, GetIcMFC) though.

I did not get far looking into Minipro.exe. I used IDA Free in Wine - this does not have the Python scripting support , but I am not going to spend $800 on IDA Pro ;-). Valentin's hint about DeviceIoControl leads to only three functions - but those in turn are called by dozens of other functions. I started looking at some of them, but quickly got tired trying to make sense of the disassembly. More practical (and I think this is what Valentin meant) would be to use the debugger, set breakpoints and then and perform specific operations and trace the execution.

[DavidGriffith]

@chenz, can you make any sense of the other stuff not referenced at all? Is it not referenced by our program or referenced neither by ours nor the official software? It seems that the device_t struct has provisions for storing that fuse information, but it wasn't ever used.

I think the next step is to come up with a cleaner script for poking through and dumping useful things from Minipro.exe and InfoIC.dll. I have some ideas that I'm playing with now (in Perl). Ultimately I'd like to get it to spit out something with the format of devices.h.

[DavidGriffith]

http://www.nullsecurity.org/i/showoff/parse_InfoIC.c

[DavidGriffith]

In GitLab by @david.rysk on Apr 25, 2018, 13:55

This work might be useful here: http://www.nullsecurity.org/article/minipro_reverse_engineering_the_infoic_dll

Looks like someone reimplemented the device database with an external .xml file where device definitions are stored.

[DavidGriffith]

Created by: chenz

This is all old news - see post 1 of this issue. And I can only repeat myself - you cannot read the whole device table from the DLL without executing it's init code.

[DavidGriffith]

In GitLab by @david.rysk on Apr 28, 2018, 14:04

Here's a modified version of the parser to work with v6.71 of the software.

https://gist.github.com/d235j/24ed26a3ec8740517d953a04155092a0

[DavidGriffith]

Created by: ozpaulb

as chenz has pointed out, the DLL's device table is partially initialized at runtime. I see (in 6.71) that some of the entries for Atmel, LATTICE, Microchip devices are filled-in at runtime.

If you run (for example) the parse_InfoIC.c on the DLL "as-is", you'll find a lot of devices with 'empty' info (names are empty strings, device info fields are all zero). That's because they haven't been initialized by the DLL startup code yet.

So, you really need to load the DLL under Windows/Wine and ensure that the init functions are called that initialize those device entries.

The "write_unlock" value can be derived from "opts3". In the current "devices.h" in git, there are 154 unique "opt3" values that have corresponding "write_unlock" values. So, a lookup table can be generated from this (assuming no new "opt3" values exist in future DLLs).

If new/unique "opt3" values are used in future DLLs, then the "write_unlock" value will need to be recalculated from the info in the EXE as suggested.