away3d / away3d-core-fp11

Away3D engine for Flash Player 11
http://www.away3d.com
Other
641 stars 276 forks source link

view.mousePicker = PickingType.SHADER Issues. #537

Closed JohnBrookes closed 11 years ago

JohnBrookes commented 11 years ago

Shader details for normals fail with shader picker on some faces.

Ive tried rolling back through dev branch commits and this issue seems to always have been so.

Test code https://gist.github.com/JohnBrookes/c3073e5434199ba691ac

JohnBrookes commented 11 years ago

So... somewhere in ShaderPicker getApproximatePosition func (line 281) Theres some rounding happening.

Load up that example and pick the top of the small cube. The localPosition of the Y is retruned as 99.99999999999997 and not 100.

This means it fails the withinBounds check (line370 shaderPicker) and localhitNormals are not set.

JohnBrookes commented 11 years ago

Maybe ShaderPicker 310-312 change to _localHitPosition.x = Math.round((((col >> 16) & 0xff) / (scX * 255) - offsX) * 1000000) / 1000000; _localHitPosition.y = Math.round((((col >> 8) & 0xff) / (scY * 255) - offsY) * 1000000) / 1000000; _localHitPosition.z = Math.round(((col & 0xff) / (scZ * 255) - offsZ) * 1000000) / 1000000; Seems to work.

DerSchmale commented 11 years ago

Just a classic floating point problem. The correct way is to fix the bound checks to use epsilon. Rounding is expensive and can even make things works with some values; you're just lucky with a magic 100 number ;)

rob-bateman commented 11 years ago

fixed in 425fb0bdecaea56d7e892c8b55c50a432168bab1