thewhiteninja / ntfstool

Forensics tool for NTFS (parser, mft, bitlocker, deleted files)
MIT License
483 stars 97 forks source link

[Request] Provide option to interpret index record structures in a more explicit way #4

Closed lonecrane closed 3 years ago

lonecrane commented 3 years ago

Dear author, NTFS employs MFT records and index records to organize directories. Ntfstool can extract single given file from a volume, demonstrating that it fully supports interpreting index records (in an implicit way). However, when it comes to the situation of fixing a corrupted file system (where we need to check whether all the index entries are valid), it is necessary to produce more explicit results than present, to help check the index chains (especially for the layman only knowing some basic concept about NFTS such as me). In other words, what I am looking forward to is actually to find out where the B+ tree breaks step-by-step. For example, via the option mft disk=x volume=y inode=5, I can get the understandable information of the MFT file record related to the root directory as below. However, I cannot track its sub files and subdirectories further according to this result directly because I don't know how to use the obtained 'First Entry Offset' (which is 16) to check further, and don’t know the detailed information of index nodes, whose length is 152.

| 4  | $INDEX_ROOT            | False        | 168    | Attribute Type          : Filename                         |
|    |                        |              |        | Collation Rule          : 1                                |
|    |                        |              |        | Index Alloc Entry Size  : 4096                             |
|    |                        |              |        | Cluster/Index Record    : 1                                |
|    |                        |              |        | -----                                                      |
|    |                        |              |        | First Entry Offset      : 16                               |
|    |                        |              |        | Index Entries Size      : 152                              |
|    |                        |              |        | Index Entries Allocated : 152                              |
|    |                        |              |        | Flags                   : Large Index                      |
+------------------------------------------------------------------------------------------------------------------+

Hence, maybe a low-level but more explicit way for interpreting index records is needed. For example, an additional option for a given index buffer can be introduced as below: ntfstool index_buffer disk=x volume=y cluster=z where z is the starting cluster of the non-resident index record (maybe the file size is also needed), should be pointed out by a specific previous-step instruction. What’s more, as mentioned above, I want ntfstool explicitly show the detailed information of the resident index nodes when interpreting the attribute of $INDEX_ROOT and explicitly show the starting address list for all the non-resident index buffers when interpreting the attribute of $INDEXALLOCATION (I am not sure this statement is correct, but you, a smart man, can catch it ^^). As I am totally a layman, there must be some erroneous and ambiguous statement in this enhancement request. To make my request clearer, my story is that I have been working on an external 4T hard disk which had suddenly became corrupted as RAW format. Ntfstool is the most powerful tool I have found to print formatted information of various complicated NTFS binary data. With the understandable formatted text, I can focus on figuring out what happened on the hard disk whose file system is corrupted, only being required to know some basic concept about NTFS. It helps me get rid of repeatedly understand binary data and allow comparing the obtained formatted information of specific data to those of a healthy one. So far I have managed to the problem that some MFT records are dis-ordered (This problem is found with the help of ntfstool). However, the root directory is corrupted as well, and I am confused when verify the index chains. That's why I submit this enhancement request. The file record for the corrupted root directory is shown below:


MFT (inode:5) from \\.\PhysicalDrive5 > Volume:1
------------------------------------------------

Signature         : FILE
Update Offset     : 48
Update Number     : 3
$LogFile LSN      : 2099480808
Sequence Number   : 5
Hardlink Count    : 1
Attribute Offset  : 56
Flags             : In_use | Directory
Real Size         : 784
Allocated Size    : 1024
Base File Record  : 0000000000000000h
Next Attribute ID : 16
MFT Record Index  : 5
Update Seq Number : 361
Update Seq Array  : 01d50000

Attribute $INDEX_ALLOCATION not foundAttributes:
-----------

+------------------------------------------------------------------------------------------------------------------+
| Id | Type                   | Non-resident | Length | Overview                                                   |
+------------------------------------------------------------------------------------------------------------------+
| 1  | $STANDARD_INFORMATION  | False        | 48     | File Created Time       : 2019-06-12 22:14:38              |
|    |                        |              |        | Last File Write Time    : 2021-06-19 09:48:03              |
|    |                        |              |        | FileRecord Changed Time : 2021-06-19 09:48:03              |
|    |                        |              |        | Last Access Time        : 2021-06-19 23:50:07              |
|    |                        |              |        | Permissions             :                                  |
|    |                        |              |        |   read_only     : 0                                        |
|    |                        |              |        |   hidden        : 1                                        |
|    |                        |              |        |   system        : 1                                        |
|    |                        |              |        |   device        : 0                                        |
|    |                        |              |        |   normal        : 0                                        |
|    |                        |              |        |   temporary     : 0                                        |
|    |                        |              |        |   sparse        : 0                                        |
|    |                        |              |        |   reparse_point : 0                                        |
|    |                        |              |        |   compressed    : 0                                        |
|    |                        |              |        |   offline       : 0                                        |
|    |                        |              |        |   not_indexed   : 0                                        |
|    |                        |              |        |   encrypted     : 0                                        |
|    |                        |              |        | Max Number of Versions  : 0                                |
|    |                        |              |        | Version Number          : 0                                |
+------------------------------------------------------------------------------------------------------------------+
| 2  | $ATTRIBUTE_LIST        | False        | 296    | $STANDARD_INFORMATION                                      |
|    |                        |              |        | Record Num: 0005000000000005                               |
|    |                        |              |        | ------                                                     |
|    |                        |              |        | $FILE_NAME                                                 |
|    |                        |              |        | Record Num: 0005000000000005                               |
|    |                        |              |        | ------                                                     |
|    |                        |              |        | $OBJECT_ID                                                 |
|    |                        |              |        | Record Num: 0005000000000005                               |
|    |                        |              |        | ------                                                     |
|    |                        |              |        | $SECURITY_DESCRIPTOR                                       |
|    |                        |              |        | Record Num: 000400000000cac9                               |
|    |                        |              |        | ------                                                     |
|    |                        |              |        | $INDEX_ROOT                                                |
|    |                        |              |        | Name      : $I30                                           |
|    |                        |              |        | Record Num: 0005000000000005                               |
|    |                        |              |        | ------                                                     |
|    |                        |              |        | $INDEX_ALLOCATION                                          |
|    |                        |              |        | Name      : $I30                                           |
|    |                        |              |        | Record Num: 000400000000cac9                               |
|    |                        |              |        | ------                                                     |
|    |                        |              |        | $BITMAP                                                    |
|    |                        |              |        | Name      : $I30                                           |
|    |                        |              |        | Record Num: 000400000000cac9                               |
|    |                        |              |        | ------                                                     |
|    |                        |              |        | $LOGGED_UTILITY_STREAM                                     |
|    |                        |              |        | Name      : $TXF_DATA                                      |
|    |                        |              |        | Record Num: 0005000000000005                               |
+------------------------------------------------------------------------------------------------------------------+
| 3  | $FILE_NAME             | False        | 68     | Parent Dir Record Index : 5                                |
|    |                        |              |        | Parent Dir Sequence Num : 5                                |
|    |                        |              |        | File Created Time       : 2019-06-12 22:14:38              |
|    |                        |              |        | Last File Write Time    : 2019-06-12 22:14:38              |
|    |                        |              |        | FileRecord Changed Time : 2019-06-12 22:14:38              |
|    |                        |              |        | Last Access Time        : 2019-06-12 22:14:38              |
|    |                        |              |        | Allocated Size          : 0                                |
|    |                        |              |        | Real Size               : 0                                |
|    |                        |              |        | ------                                                     |
|    |                        |              |        | NameType                : DOS & WIN32                      |
|    |                        |              |        | Name                    : .                                |
+------------------------------------------------------------------------------------------------------------------+
| 4  | $OBJECT_ID             | False        | 16     | Object Unique ID        : {663f6f9a-a036-11e9-a414-28b2bdd |
|    |                        |              |        |                           e2834}                           |
+------------------------------------------------------------------------------------------------------------------+
| 5  | $INDEX_ROOT            | False        | 56     | Attribute Type          : Filename                         |
|    |                        |              |        | Collation Rule          : 1                                |
|    |                        |              |        | Index Alloc Entry Size  : 4096                             |
|    |                        |              |        | Cluster/Index Record    : 1                                |
|    |                        |              |        | -----                                                      |
|    |                        |              |        | First Entry Offset      : 16                               |
|    |                        |              |        | Index Entries Size      : 40                               |
|    |                        |              |        | Index Entries Allocated : 40                               |
|    |                        |              |        | Flags                   : Large Index                      |
+------------------------------------------------------------------------------------------------------------------+
thewhiteninja commented 3 years ago

Check the last commit and README.md#btree It's not perfect but it gives a better idea of the index parts with raw addresses. But I didn't test it on corrupted data :)

lonecrane commented 3 years ago

Thanks a lot. I am using the latest version to go further.

It seems that there is a small mismatch in a certain 'Raw address' between the analysis result and the actual one (Inspected by winhex). In a case study, the analysis result by ntfstool is shown below: +------------------------------------------------------------------------------------------------+ | VCN | Raw address | Size | Entries | +------------------------------------------------------------------------------------------------+ | 000000000000h | 000000024000h | 000000001000h | 000000000004: $AttrDef | | 000000000001h | 000000025000h | 000000001000h | 0000000000ab: New Text Document - Copy (15) - | +------------------------------------------------------------------------------------------------+ | 000000000002h | 00000058a000h | 000000001000h | 0000000000b8: New Text Document - Copy (28) - | | 000000000003h | 00000058b000h | 000000001000h | 000000000090: New Text Document - Copy (103).t | | 000000000004h | 00000058c000h | 000000001000h | 0000000000c4: New Text Document - Copy (40) - | | 000000000005h | 00000058d000h | 000000001000h | 0000000000d1: New Text Document - Copy (53) - | | 000000000006h | 00000058e000h | 000000001000h | 0000000000de: New Text Document - Copy (66) - | | 000000000007h | 00000058f000h | 000000001000h | 0000000000eb: New Text Document - Copy (79) - | | 000000000008h | 000000590000h | 000000001000h | 000000000104: New Text Document - Copy (104) - | | 000000000009h | 000000591000h | 000000001000h | 0000000000f7: New Text Document - Copy (91) - | | 00000000000ah | 000000592000h | 000000001000h | 00000000003d: New Text Document - Copy (20).tx | | 00000000000bh | 000000593000h | 000000001000h | 00000000004a: New Text Document - Copy (33).tx | | 00000000000ch | 000000594000h | 000000001000h | 000000000057: New Text Document - Copy (46).tx | | 00000000000dh | 000000595000h | 000000001000h | 000000000064: New Text Document - Copy (59).tx | | 00000000000eh | 000000596000h | 000000001000h | 000000000070: New Text Document - Copy (71).tx | | 00000000000fh | 000000597000h | 000000001000h | 00000000007d: New Text Document - Copy (84).tx | | 000000000010h | 000000598000h | 000000001000h | 00000000010a: New Text Document - Copy (110) - |

However, the second node is actually located in 000000590000h, which can be verified by winhex. And the runlist data is '11 01 24 21 13 65 05', demonstrating that the nodes should consists of 1 node and 0x13 continuous nodes. In other words, the first run is with the length of 1 other than 2. Thus, the analysis result is slightly different from the acutal case.

thewhiteninja commented 3 years ago

I got the same bug with enough files. I pushed a quick fix and print dataruns for index_allocations attribute.