Open NachoLibrador opened 10 years ago
You mention that when you read the physical file, the slot offset starts at 478. But what physical file are you reading? Are you reading the backup file itself (.bak)? If so - how are you doing that? OrcaMDF is only made to work on MDF files.
What are you using to read the data file, Database
or RawDatabase
?
Both backup file and mdf file restored from the backup reads 478 instead of 222. MDF file attached to MSSQL with DBCC PAGE([database], 1, 1560, 3) reads 222 in the output of the command even though a direct physical read of the same MDF reads 478.
From: Mark S. Rasmussen [mailto:notifications@github.com] Sent: Saturday, March 22, 2014 5:21 PM To: improvedk/OrcaMDF Cc: NachoLibrador Subject: Re: [OrcaMDF] data page type = 1. Slot Offset is read wrong (#29)
You mention that when you read the physical file, the slot offset starts at 478. But what physical file are you reading? Are you reading the backup file itself (.bak)? If so - how are you doing that? OrcaMDF is only made to work on MDF files.
What are you using to read the data file, Database or RawDatabase?
— Reply to this email directly or view it on GitHub https://github.com/improvedk/OrcaMDF/issues/29#issuecomment-38369100 . https://github.com/notifications/beacon/7034888__eyJzY29wZSI6Ik5ld3NpZXM6QmVhY29uIiwiZXhwaXJlcyI6MTcxMTE1MzI1NiwiZGF0YSI6eyJpZCI6MjgzMDYwNTF9fQ==--c24b225824bfb6517e9637465b28bae087e0cd1f.gif
does it matter? You read offsets of any king of page for any moethod from the end by 2 bytes as many times as slots count in the header and the method is namespace OrcaMDF.Core.Engine.Pages
private void parseSlotArray(){
for (int i = 0; i < Header.SlotCnt; i++) SlotArray[i] = BitConverter.ToInt16(RawBytes, RawBytes.Length - i * 2 - 2);
and BitConverter.ToInt16(RawBytes, RawBytes.Length - 0 * 2 - 2); return 478 2bytes there 222 and 1 if there would be 222 and 0 it would give 222. I will prepare example mdf file and will shrink it to as much as possible to reproduce the problem
I would think it is a partial write during backup. So there suppose to be uncommited kind of transaction sopmewhere in the bakup file that generated MDF in a log or in the page itself and i am not sure where to look for it
I have seen similar stuff happen when I took at 2008 database and restored it in a 2008 R2 server. One bit in the slot array flipped when I looked at DBCC PAGE, but the on-disk structure had the opposite bit. I never found out why, but all data was still there so I didn't have much incentive to really look into it.
Let me ask another question - are you missing data or is this more of an academical adventure? Is the database a 2008 R2 version? The OrcaMDF Database class (which I can see you're using) only supports straight 2008 R2 databases, whereas RawDatabase will work on 2005-2012, but only parses the physical structures and nothing more.
Yes i do miss data not physicly but I can not map it. Say if slot number is wrong then I start reading record at slot 0 at address 478 instead of 222. So record is wrong and I have a lot of thouse. Also say you have 2 fields Name and Last Name, varchar and they are written at the end of record in this way: MarkRasmussen. If your last name adddress is shifted by one bit say instead of reading stars at index 5 it would say 4 or 6 you would get your last name to be either kRasmussen or asmussen. So it is a big problem. Another case: if bit is missplaced in your name you might get instead of Mark, Lark or Nark, basicly + - 1 on your next letters. The only positive thing I see that missplaced bit happens only once on the page. I know that this is for databases that have havy multiuser use so they do lots of IO operations. I looked into Tornbit in the header trying to see if info how to fix it there but could not make anything of it. No Torn Page error is raised and db reads data clean so the info how to fix thew page is there either in mdf or in log file but I am not sure. COuld be ven in the page itself.
the database is 2005
Hi Mark, There is no func on the website to upload database files so I am just sending you them directly.
I simplified the database removed all client sensitive data so many errors got fixed when I shrunk the database. But I am lucky there is still one there that shows the problematic behavior. Here are the cords:
ObjectId = 245575913 Page Type = 1 Page Id = 1086 Wrong Slot = 0 reads as 322 should be 834 page start index in mdf file = 8896512 slot read index in mdf file = 8904702 BitConverter.ToInt16(DataBytes, 8904702); gives 322
When attached to SQL server data displayed correctly with no errors:
DBCC TRACEON(3604) DBCC PAGE([7612ce8d-cc17-4e40-b3ba-de75ce52eee9], 1, 1086, 1)
Shows: OFFSET TABLE:
Row - Offset
59 (0x3b) - 6445 (0x192d)
58 (0x3a) - 6244 (0x1864)
57 (0x39) - 6152 (0x1808)
56 (0x38) - 6080 (0x17c0)
55 (0x37) - 5978 (0x175a)
54 (0x36) - 5880 (0x16f8)
53 (0x35) - 5795 (0x16a3)
52 (0x34) - 5675 (0x162b)
51 (0x33) - 5574 (0x15c6)
50 (0x32) - 5497 (0x1579)
49 (0x31) - 5374 (0x14fe)
48 (0x30) - 5279 (0x149f)
47 (0x2f) - 5196 (0x144c)
46 (0x2e) - 5092 (0x13e4)
45 (0x2d) - 4985 (0x1379)
44 (0x2c) - 4901 (0x1325)
43 (0x2b) - 4829 (0x12dd)
42 (0x2a) - 4715 (0x126b)
41 (0x29) - 4638 (0x121e)
40 (0x28) - 4566 (0x11d6)
39 (0x27) - 4454 (0x1166)
38 (0x26) - 6591 (0x19bf)
37 (0x25) - 4194 (0x1062)
36 (0x24) - 4122 (0x101a)
35 (0x23) - 4050 (0xfd2)
34 (0x22) - 3978 (0xf8a)
33 (0x21) - 3906 (0xf42)
32 (0x20) - 3834 (0xefa)
31 (0x1f) - 3738 (0xe9a)
30 (0x1e) - 3596 (0xe0c)
29 (0x1d) - 3496 (0xda8)
28 (0x1c) - 3424 (0xd60)
27 (0x1b) - 3344 (0xd10)
26 (0x1a) - 3254 (0xcb6)
25 (0x19) - 3144 (0xc48)
24 (0x18) - 3043 (0xbe3)
23 (0x17) - 2959 (0xb8f)
22 (0x16) - 2869 (0xb35)
21 (0x15) - 2788 (0xae4)
20 (0x14) - 2698 (0xa8a)
19 (0x13) - 2611 (0xa33)
18 (0x12) - 2539 (0x9eb)
17 (0x11) - 2445 (0x98d)
16 (0x10) - 2358 (0x936)
15 (0xf) - 2248 (0x8c8)
14 (0xe) - 2165 (0x875)
13 (0xd) - 2082 (0x822)
12 (0xc) - 2010 (0x7da)
11 (0xb) - 1937 (0x791)
10 (0xa) - 1811 (0x713)
9 (0x9) - 1739 (0x6cb)
8 (0x8) - 1613 (0x64d)
7 (0x7) - 1541 (0x605)
6 (0x6) - 1428 (0x594)
5 (0x5) - 1341 (0x53d)
4 (0x4) - 1269 (0x4f5)
3 (0x3) - 1197 (0x4ad)
2 (0x2) - 1060 (0x424)
1 (0x1) - 918 (0x396)
0 (0x0) - 834 (0x342)
thoose are coords of: SELECT * from Vehicle WHERE VehicleId = 445 And if read with orcamdf you will not get that record at all even though if you attach database you will see it there. The file has also examples of wrong varchar mapping and so on but lets just find how to fix wrong slot i am sure it would fix the rest of the problems.
so now what I found is that if you update that record any field you wil lget it fixed in the mdf file. Slot will become 834. So it is there pending somewhere in the db files the question is where and how to read it.
322 -> 0100001010000000 834 -> 0100001011000000 it is always only 1 bit difference
basicly that makes yours and mine approach of reading direct data useless in a real enviroment. That is why i am thinking it is important to fix it and find out what is going on. I had same thing i saw it once in a test file and thought it will never happen enogth to make it uselees. But now I have a datbase on my hands that has over 1K of errors of this type and even htough I read 99.9 % of the data enability of reading 100% makes all 99.9% effort useless :(
it is like one would charge a client $5 for cup of coffee and everything works OK but once a day one would charge 1mil dollars accidentely on their credit card even though one put $5 in for the price, that makes operations impossible :(
Hi Mark,
I have a question regarding ORCAMDF project and particularly about reading simple records data page pagetype = 1.
I have a full backup file for mssql database.
There is a page index 1560 in the file and it has 50 slot offsets (50 records in it). When I read physical page from the file first slot offset reads as 478. When I restore the backup and run: DBCC PAGE([database], 1, 1560, 3) it reads first slot offset as 222. MDF file in restored database still reads first slot as 478 even though server somehow knows it is 222. I only get to see 222 on the page if I do an update to that page: UPDATE Vehicle SET VIN = ‘test’ WHERE Vehicle ID = 1433 Now I do see 222 in mdf file.
I have a lot of fields like that with just 1 bit difference in them: 478 -> 0111101110000000 222 -> 0111101100000000
Only 1 bit difference. I understand it is because that bit did not get updated during the backup time but was logged somehow in the backup file. Would you help me to find where I can get the differential info in the backup? In other words how can I fix my data page with the info included in the backup.
I can send you the full backup file if you need it too.
Thank you very much,