search

hydradarpa / kalman-hydra

Python and C++ code for Hydra optical flow, behavior analysis with extended Kalman filter tracker
2 stars 0 forks source link

Flow data only stored to 8 digit precision on GPU? #6

Closed benlansdell closed 8 years ago

benlansdell commented 8 years ago

See test_cuda.TestCUDA_ones.test_get_pixel_data()

benlansdell commented 8 years ago

Not sure why this would be, or what effect it has...

benlansdell commented 8 years ago

This is to be expected. Stored as float32, which stores:

"float: 32 bits (4 bytes) where 23 bits are used for the mantissa (6 to 9 decimal digits, about 7 on average). 8 bits are used for the exponent, so a float can “move” the decimal point to the right or to the left using those 8 bits. Doing so avoids storing lots of zeros in the mantissa as in 0.0000003 (3 × 10-7) or 3000000 (3 × 107). There is 1 bit used as the sign bit."

http://stackoverflow.com/questions/13542944/how-many-significant-digits-have-floats-and-doubles-in-java