search

datamapper / do

DataObjects
147 stars 73 forks source link

postgres - precision lost when casting on double precision #63

Open donhill opened 11 years ago

donhill commented 11 years ago

I have a table that is using a type of double precision. I am trying to perform a round(cast(column as numeric),2). When I perform this in psql I get the correct results. So 17.1799913573964 becomes 17.18. When I am using the datamapper postgre driver I get the value 0.1718E2. Any thoughts on this. Here is the psql query.

select id, round(cast(distance as numeric),2) miles, distance from (SELECT id, ( 3959 * acos( cos( radians(42.0926245) ) cos( radians( lat ) ) * cos( radians( lng ) - radians(-88.1152973) ) + sin( radians(42.0926245) ) \ sin(radians( lat ) ) ) ) AS distance FROM operator_positions ) position where distance < 20

dbussink commented 11 years ago

I don't really understand the issue. 0.1718E2 seems to be the same as 17.18 to me.

dkubb commented 11 years ago

Me too:

2.0.0p247 :001 > 0.1718E2 == 17.18
 => true