-
in `isInvertible`, determinant is compared directly with zero. there is a very good chance this value will be floating point, so we should define a class `EPSILON` around `1E-10` for value comparison…
-
I'm trying to use pow and running into link errors (undefined reference to 'pow'). What flags do I need to change to get floating point math working?
-
| | |
|--------------------|----|
| Bugzilla Link | [PR17988](https://bugs.llvm.org/show_bug.cgi?id=17988) |
| Status | NEW |
| Importance | P normal |
|…
-
Hi,
I am trying to compare two completed genomes, but mummerplot consistently given the following error:
"integer overflow; change to floating point"
Thanks.
Xingbo
-
I want to support the ESP32 platform, which does use a Floating Point Unit (FPU), but it's very slow. It would be beneficial to therefore replace with integer arithmetic, for embedded systems with no …
-
Floating point numbers are known to be an imperfect model of real numbers, causing rounding errors in calculations and difficulty for equality comparison.
For example, the Euclidean length example …
louy2 updated
2 years ago
-
Presently the only types available in TMVM are Ints. and unsigned single-byte Ints at that. support for floating-point operations would greatly improve the usefulness of the machine.
-
This was suggested as an idea from #348 by @brianharvey. I definitely think it's worth considering, so I'm going to post this here (since the other issue is closed).
Here's what was said:
@brianharv…
-
There are challenges when using `BigDecimal` and constructors such as `BigDecimal(Double)` _vs_ `BigDecimal(String)`. For example: `BigDecimal(1.0)` has different "scale" than `BigDecimal("1.0")`
…
-
I'm having issues with a halide being a little too aggressive in it's handling of floating point calculations.
The issue at its core is that halide thinks that transforming `(x+1)*n` into `(x*n)+n…