Closed yakra closed 6 years ago
What was that thing that happened when GISplunging PEI that one time? • A possible instance of this bug having detectable effects? • Was I using a trimmed DBF? • Why did SegDump work, rather than skipping file per Record number mismatch? • Can I recreate this and test?
24fc2346be8c476383ed593ac71d0837-260.wpt
Similarly:
ShapeRoot /home/yakra/gis/data/tx/txdot-roads_48113/txdot-2015-roadways_48113
KeyField RTE_UNIQ_I
KeyValue 508
Should SegDump record 1; SegDumps record 4 This segment has RTE_UNIQ_I = 1154.
Unmolested shapefile, as downloaded from, uhh... wherever. (Hell, in this case, why does GISplunge even work right some of the time?)
PE, original DBF file:
ShapeRoot /home/yakra/gis/data/pe/nrn_rrn_pe_12.0_shp_en/NRN_PE_12_0_ROADSEG
KeyField NID
KeyValue 24fc2346be8c476383ed593ac71d0837
Dumps record 260, as expected. Correct NID value.
260-2 http://www.openstreetmap.org/?lat=46.272151&lon=-63.118827
260-1 http://www.openstreetmap.org/?lat=46.274389&lon=-63.128079
PE, trimmed DBF (10.2 MB), 20171026 thru 20171105, and PE, trimmed DBF (9.8 MB), 20171106 thru 20171110:
ShapeRoot /home/yakra/gis/data/pe/nrn_rrn_pe_12.0x_shp_en/NRN_PE_12_0_ROADSEG
KeyField NID
KeyValue 24fc2346be8c476383ed593ac71d0837
Should SegDump record 260; SegDumps record 262 This segment has NID = a1bf9edc8c324b4d8ffdbc7669583dfd.
262-2 http://www.openstreetmap.org/?lat=46.708760&lon=-64.215424
262-1 http://www.openstreetmap.org/?lat=46.708702&lon=-64.215755
ProcRte
566 DBF.seekg(HeaderLength + KeyOffset + 1);
--> DBF.seekg(577 + 79 + 1);
--> DBF.seekg(657);
569 if (KeyMatch(KeyValue, DBFname, DBF.tellg()))
--> if (KeyMatch("508", DBFname, 657))
KeyMatch
515 for (int index = 0; index < strlen(KeyValue); index++)
--> for (int index = 0; index < strlen("508"); index++)
--> for (int index = 0; index < 3; index++)
// "508" matches, followed by 0x20
523 return 1;
ProcRte
569 KeyMatch("508", DBFname, 657)
== 1
570 int RecordNum = DBF.tellg()/RecordLength+1;
--> int RecordNum = 657/199+1;
--> int RecordNum = 3+1;
--> int RecordNum = 4;
BOOM! Note: HeaderLength > RecordLength
targeting record 1 this time
566 DBF.seekg(HeaderLength + KeyOffset + 1);
--> DBF.seekg(1569 + 0 + 1);
--> DBF.seekg(1570);
570 int RecordNum = DBF.tellg()/RecordLength+1;
--> int RecordNum = 1570/2029+1;
--> int RecordNum = 0+1;
--> int RecordNum = 1;
Ayuh. Note: HeaderLength < RecordLength And THAT is how I failed to notice this bug for 7 years.
targeting record 1 this time
566 DBF.seekg(HeaderLength + KeyOffset + 1);
--> DBF.seekg(1569 + 0 + 1);
--> DBF.seekg(1570);
570 int RecordNum = DBF.tellg()/RecordLength+1;
--> int RecordNum = 1570/593+1;
--> int RecordNum = 2+1;
--> int RecordNum = 3;
Boom. Note: HeaderLength > RecordLength
TX
570 int RecordNum = (DBF.tellg()-HeaderLength)/RecordLength+1;
--> int RecordNum = (657-577)/199+1;
--> int RecordNum = 80/199+1;
--> int RecordNum = 0+1;
--> int RecordNum = 1;
PE, original
570 int RecordNum = (DBF.tellg()-HeaderLength)/RecordLength+1;
--> int RecordNum = (1570-1569)/2029+1;
--> int RecordNum = 1/2029+1;
--> int RecordNum = 0+1;
--> int RecordNum = 1;
PE, trimmed
570 int RecordNum = (DBF.tellg()-HeaderLength)/RecordLength+1;
--> int RecordNum = (1570-1569)/593+1;
--> int RecordNum = 1/593+1;
--> int RecordNum = 0+1;
--> int RecordNum = 1;
gisplunge02.cpp, in ProcRte(), line 570:
int RecordNum = DBF.tellg()/RecordLength+1;
does not account for DBF header length. Probably gets the correct INT most of the time anyway, due to RecordLength > HeaderLength.~~Hex editor: create a file with RecordLength < 32. • See if I can make it rain cheese (Record number mismatch)~~
• Remind myself why I started numbering at 1, and not 0. Could it be this? Line 585:
if (RecordNum != SHP.get()*0x1000000 + SHP.get()*0x10000 + SHP.get()*0x100 + SHP.get())
Remind myself what exactly's stored at this point in the SHP file.