yakra / tmtools

Tools to aid in development of the TravelMapping project
0 stars 0 forks source link

DBFtrim: MinEx0, MaxIntD, DecCount & pals, and differing sig figs #28

Closed yakra closed 6 years ago

yakra commented 6 years ago

Relevant to how things got this way: https://github.com/yakra/tmtools/issues/6#issuecomment-370327880 https://github.com/yakra/tmtools/issues/23#issuecomment-372090341

field::GetMax cannot cope with input values with differing sig figs.

Suppose DecCount = 2, and Input values are 0.0 0.00 1.0 1.00 10.0 10.00 12.0 12.00 if the 1st value is 0.0, MinEx0 gets set to 1. This is still < DecCount, so MinEx0 is not increased to account for the '.' itself. if the 2nd value is 0.00, MinEx0 is already 1, so nothing more happens. Output values are 0. 0.0 1. 1.0 10. 10.0 12. 12.0

Simply detecting the '.' when trimming, rather than comparing MinEx0 against DecCount, will not work, come file writing time: outDBF.write(fVal, tDBF.fArr[fNum].len-fI+vlen-oDBF.fArr[fNum].MinEx0); resolves to outDBF.write(fVal, 2-2+3-3); or outDBF.write(fVal, 3-3+3-3); or outDBF.write(fVal, 2*(3-3));... You get the idea.

Goals:

new size = MaxIntD+1+MaxDecP or old size, whichever is smaller...

yakra commented 6 years ago

Pseudocode...

// Trimming...
    // bye-bye MinEx0

// Comparing...
if (tDBF.fArr[fNum].DecCount) tDBF.fArr[fNum].len = MaxIntD+1+tDBF.fArr[fNum].DecCount;
else if (tDBF.fArr[fNum].len < MaxIntD) tDBF.fArr[fNum].len = MaxIntD;
if (tDBF.fArr[fNum].len > len) tDBF.fArr[fNum].len = len;

// Saving...
if (vlen > strchr(fVal, '.') - fVal + tDBF.fArr[fNum].DecCount + 1)
    vlen = strchr(fVal, '.') - fVal + tDBF.fArr[fNum].DecCount + 1;
for (fI = tDBF.fArr[fNum].len-vlen; fI < tDBF.fArr[fNum].len; fI++) outDBF.put(' ');
outDBF.write(fVal, vlen);
yakra commented 6 years ago

~/gis/data/tests/dec-ints.dbf ~/gis/data/tests/dec-ints2.dbf ~/gis/data/tests/dec-ints3.dbf all behaving as expected

~/gis/data/tx/txdot-roads_48113/txdot-2015-roadways_48113.dbf only DIFF is DecCount in Shape_Leng field: now corrected from 11 to 10 (see https://github.com/yakra/tmtools/issues/6#issuecomment-370251373) Good!

everything else: no DIFF