jhkolb / sparkRocks

Parallel rock slicing implementation on Apache Spark
GNU General Public License v2.0
0 stars 0 forks source link

Floating Point Arithmetic #10

Open jhkolb opened 9 years ago

jhkolb commented 9 years ago

We need to figure out how to deal with the inevitable rounding and arithmetic errors that come from using floats, doubles, etc. A couple of places where this is particularly important:

jhkolb commented 8 years ago

This has become more complicated with the processor block merging code, where rounding to anything more than 4 decimal places causes the related unit tests to fail.

This seems to be because blocks that share a processor joint aren't associated with each other in the groupBy that uses normal vectors.

shellshocked2003 commented 8 years ago

I think we can close this now, since we no longer are dealing with the "processor" joint stuff. Do you agree?

jhkolb commented 8 years ago

Well, this is kind of an ongoing thing we need to keep in mind, especially if we add in more numerical calculations.

I also think we changed the code at some point so that we are rounding to different tolerances in different places and aren't sticking to some standard value like 1.0e-6.

Once we feel like our numerical calculations have reached a pretty stable point, I think we should do an audit of the code to see where we may be introducing unnecessary floating point error (e.g. dividing in the body of the loop when we could lift this out) and see if we can't stick to a standard tolerance throughout the code.

shellshocked2003 commented 8 years ago

Okay, that sounds reasonable. I think I eliminated the last spot in the code where we were using a tolerance different from 1.0e-6. Just pushed that to master this morning.