TMC Calibration Limitations

[nthallen]

TMC completely refuses to do piecewise linear conversions of uint32_t values due to the fact that there was not a handy larger type to handle rational expressions. While it would be possible to extend the approach by using an int64_t, I think an alternative strategy should be possible. Consider:

• breaking up the span into smaller stretches (1/2, 1/4)
• come up with a better error estimator that does not require brute force

SCoPEx/PropMtr/TM includes many crude work-arounds to avoid these limitations. In most cases, uint32_t types were changed to int32_t with the range arbitrarily limited.

[nthallen]

In addition, the current linear interpolation formulation is subject to intermediate overflow. I believe this can be corrected by subtracting the mean X value before multiplying.

[nthallen]

I was also unable to extend double SD_angle_t from %10.5lf to %11.6lf.

[nthallen]

I tried this hack, but it still failed:

  Calibration (SD_f_angle_t, SD_degrees) {
    -9.6,             -99999,
    -9.5,             -99999,
    -9.424777960769379, -540,
     9.424777960769379,  540,
     9.5,              99999,
     9.6,              99999 }

[nthallen]

gen_int_tcvt() uses an icvt function, then generates a tcvt function to go to text. It uses the range of the text fmt (%11.6lf) rather than the actual range of the icvt function. In this case, the fmt range would require more than 34 bits while the icvt range only requires 31 bits.