The current code for handling the Riegl coordinate transformation is set up to do the actual calculation as float32. When world coordinates are UTM, the numbers are large enough that there is a loss of precision at the sub-metre level. For TLS scans such as the Riegl gives us, we need the precision to be more like millimetre. Hence this calculation should be done as float64.
I have put in place a hack to get around it for now, as just changing to had too many implications elsewhere in the file. However, this is what ought to be done, ultimately. I have had to circumvent the use of the pylmatrix code, which is a shame.
Original comment by Sam Gillingham (Bitbucket: gillins, GitHub: gillins).
See 47aeeac. Let me know if this fixes it.
I think I did this because most of the Riegl data comes in as floats, but obviously you need to be careful when doing calculations with it.... Now use double everywhere I can.
gillins
commented
7 years ago
Original report by Neil Flood (Bitbucket: neilflood, GitHub: neilflood).
The current code for handling the Riegl coordinate transformation is set up to do the actual calculation as float32. When world coordinates are UTM, the numbers are large enough that there is a loss of precision at the sub-metre level. For TLS scans such as the Riegl gives us, we need the precision to be more like millimetre. Hence this calculation should be done as float64.
I have put in place a hack to get around it for now, as just changing to had too many implications elsewhere in the file. However, this is what ought to be done, ultimately. I have had to circumvent the use of the pylmatrix code, which is a shame.