SchrodingersGat
commented
7 years ago I like the idea of using integer metric units for internal calculations, as this confers obvious computational advantage.
I also like the idea of using double precision mm for file storage, as for the vast majority of cases, we would be representing x,y data that is close to some local origin.
Examples being footprints and symbols, which will (rarely) be larger than 9000km and most likely in the order of a few (to a few dozen) mm. So using floating point would cut down on filesize.
iromero91
Continuing the discussion on IRC, I propose the internal units in the program be metric, as inches can be represented exactly with a metric quantity with just 3 orders of magnitude more resolution (conversion factor 254 mm = 10 in), the converse is not true for imperial as the conversion factor contains a large prime number (127) which causes a very large repeating fractional number both in binary and decimal. Using a compromise unit with the factor 127 would halve the resolution in metric units, and double the resolution in imperial. but since we're going to use 64 bit numbers I feel the extra bit of dynamic range is not worth the loss of readability caused by not using real engineering units.
Talking orders of magnitude, it was proposed on IRC to use fixed point 64 bit values for representing coordinates, one possibility is to use the picometer as the smallest representable value. In a signed 64 bit value (63 bits of dynamic range) we would have a range from -9223 to 9223km and could represent quantities down to 0.01 nanoinches with mathematical exactitude, and then it could only be off by a picometer, totally eliminating roundoff problems.
Another option would be to use double precision float millimeters. Even if not used for actual calculation, that representation is handy for file IO (perhaps using decimal floats, not binary to avoid subtle errors from popping up due to base 10 and 2 not sharing all prime factors) Floats would also be necessary anyway for graphical representation as that's what opengl and most graphical back ends expect.