Inconsistent numerical results for sincos, likely connected to optimization/image caching

[philbit]

As discussed here on discourse, the call

julia --optimize=1 --project=. -e 'c = 0.46309504630950465; using Colors; display(cos(c)); display(sincos(c)[2])'

might return

0.8946741679895334
0.8946741679895333

or

0.8946741679895334
0.8946741679895334

depending on whether Colors.jl has previously been precompiled with optimization level 2 in that depot or not.

The reason for this seems to be that optimization level 2 changes the result from 0.89...334 to 0.89...333 (see the linked discourse thread on how it does that). But even when specifying --optimize=1, the cached image created for sincos with optimization level 2 is loaded, which is allowed according to this article in the docs. Consequently, the results can also be made consistent again by calling sincos once before loading Colors.

Two questions arise from this:

1. Is it ok for floating point functions to return different values depending on optimization level 1 or 2?
2. If the answer to the first question is "yes" (which I could well imagine, but am not sure about), isn't it dangerous to allow images from higher optimization levels to be loaded at lower optimization levels? This way, we could end up with a mix of different functions (perhaps this doesn't only happen for sincos), making any numerical results dependent on the entire precompilation history in that depot, i.e. non-deterministic from the standpoint of a single julia session started with certain flags (a nightmare for reproducibility).

Of course, if the answer to question 1 is no, then allowing higher-optimization-level images to be used is completely consistent and this is rather a bug in sincos.

Julia Version 1.9.3
Commit bed2cd540a1 (2023-08-24 14:43 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 12 × Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
  Threads: 1 on 12 virtual cores

[Keno]

If the function is not considered :consistent, this is not a bug. There's other things that can cause effects like this also (e.g. inlining, interpreted vs compiled). It also happens in non-julia languages. That said though, I don't see why this function necessarily needs to be non-:consistent and if it does that could probably be moved behind the fastmath flag.

[Seelengrab]

sincos is inferred :consistent though:

julia> Base.infer_effects(sincos, Tuple{Float64})
(+c,+e,!n,+t,+s,+m,+u)

I think the OP used "consistent" not in terms of effects, but in terms of observed result.

[philbit]

Indeed, I was more concerned with the result and that it depends on things (what was cached in the depot, in what order functions are being called, etc.) which I thought it definitely shouldn't depend on. Especially for a function without side effects, I would have expected image caching to be set up in such a transparent way that it doesn't have an effect on the result.

As I said, I can understand that the result might depend on all sorts of other things (optimization level, inlining, etc. as mentioned), but the same julia call with exactly the same flags should always return the same result, or not?

Now whether this is more a problem of sincos or a general problem with using cached images from higher optimization levels (as permitted by the docs) I cannot say, since I don't know what assumptions about the behavior of functions went into the decision to allow images from higher optimization levels. It could also be that other functions exhibit the same behavior and we just happened to discover it here because we were able to compare sincos and cos directly.

[gbaraldi]

I wonder where the difference is coming from here though, because if it uses fma it should still be the same. This isn't a case of SIMD vs no SIMD from a top level look.