Sensitivity to very small floating point calculation errors

[pagameba]

The code in _calculateSizes() lines 161 - 164 are very sensitive to extremely small differences. I ultimately discovered a very small rounding error in the calculation of one of my resolutions and was able to fix my problem without changing any code in Proj4Leaflet.

However, JavaScript rounding errors are well known (try 0.1 + 0.2) and it seems to me that this block of could would be quite sensitive to these small differences in the probably case that some combination of floating point numbers triggers a small floating point math error.

From what I understand of the code, the resulting values in this._sizes are expected to be exact multiples of the this.options.tileSize and perhaps Line 163-164 could be changed to round the values to nearest multiple of this.options.tileSize

[perliedman]

Thanks. you're correct, that code can be simplified somewhat, I can at least see two operations that can be eliminated in there, shortening the lines into:

sizes[i] = L.point((crsBounds[2] - crsBounds[0]) / this._scales[i],
    Math.ceil((crsBounds[3] - crsBounds[1]) / projectedTileSize)
    * projectedTileSize * this._scales[i]);

However, I think the calculation will still be sensitive to rounding problems, and it is hard to elminate completely.

In general, I've encountered so many problems with different GIS systems and rounding errors, that I always go to great lengths trying to keep to integer coordinates and resolutions, and especially making sure bounds align with the tile grid. I know GeoWebCache didn't work that well with non-integer resolutions (although, to be fair, it might have been an old version).

[pagameba]

Not quite what I was thinking ... the value of x and y in the size needs to be in some multiple of this.options.tileSize like this:

var x = (crsBounds[2] - crsBounds[0]) * this._scales[i];
var y = Math.ceil((crsBounds[3] - crsBounds[1]) / projectedTileSize)
    * projectedTileSize * this._scales[i]);
var ts = this.options.tileSize
sizes[i] = L.point( Math.round(x / ts) * ts, Math.round(y / ts) * ts);

I didn't try it but the the idea is that if x turns out to be 255.99999999999 (for example) it really needs to be rounded to exactly 256.

[perliedman]

The only bounds that are really used in a context where tile grid alignment matters, is the upper y bound. The upper x bounds is actually never used for anything, so rounding it doesn't matter (for now).

The upper y bound is already rounded with Math.ceil, so it shouldn't have to use extra rounding.

Anyway, simplified the calculation a bit in commit d42a13bebbb9f4fa2b462760cef1f83c36235e28, please re-open if you find more problems related to rounding problems.

[pagameba]

This doesn't work for me in two cases. First, my miller world projection has a level 0 resolution of 120538.13612459375 but this turns out to be one of the numbers that javascript cannot represent. This value gets modified to 120538.13612459374 which causes the upper Y value to be calculated as very slightly more than 1 and the use of Math.ceil means that we get 2 rows of tiles requested instead of just one. I changed Math.ceil to Math.round to avoid this problem.

I had a similar problem with the x value but I don't have the details at hand. I was a bad boy and didn't fork/commit my changes, but here is the code that I am currently using and it works in all my cases (after fixing a bug in proj4.js for eqc projections)

projectedTileSize = tileSize / this._scales[i];
upperY = crsBounds[1] + Math.round((crsBounds[3] - crsBounds[1]) /                                          projectedTileSize) * projectedTileSize;
x = Math.round(((crsBounds[2] - crsBounds[0]) * this._scales[i])/tileSize) * tileSize;
y = Math.round(((upperY - crsBounds[1]) * this._scales[i])/tileSize) * tileSize;
sizes[i] = L.point(x,y);

This ensures that the sizes are exact multiples of this.options.tileSize, which makes Leaflet happy. As this code only gets executed once and sizes are cached, I don't think the extra overhead of the rounding will have any performance impact.

[perliedman]

Hm, I see. The problem with just replacing ceil with round is that a common case is to have bounds that misalign with the grid with say 0.25 tiles. In this case, rounding will actually cut the bounds one tile short (I actully did that when I first implemented the grid alignment).

I guess we could check if the limit is within a small distance to the closest integer (which would be your case), and use round. Otherwise, use ceil like now. Maybe there's a more elegant way to accomplish that?

[pagameba]

parseFloat(n.toPrecision(6)) perhaps? This should effectively round at 6 decimal places of precision. Some other level of precision could be used of course. I think I've only seen effects around 14-16 decimal places - its hard to count all those zeros :)

[perliedman]

Sorry for taking so long, but this should be fixed by 5109b3ad09ebb0f6e0d474b63c05e818ccf22ae6.