ErrorException with in-place updating with function wrapper specialization in debugger environments

[markowkes]

Summary - Running the following code through the debugger in VS Code fails with Error Exception: `llvmcall` must be compiled to be called

MWE -

using DifferentialEquations

function lorenz!(du,u,p,t)
    du[1] = 10.0*(u[2]-u[1])
    du[2] = u[1]*(28.0-u[3]) - u[2]
    du[3] = u[1]*u[2] - (8/3)*u[3]
end

function lorenz(u,p,t)
    du=similar(u)
    du[1] = 10.0*(u[2]-u[1])
    du[2] = u[1]*(28.0-u[3]) - u[2]
    du[3] = u[1]*u[2] - (8/3)*u[3]
    return du
end

u0 = [1.0;0.0;0.0]
tspan = (0.0,100.0)

# Try using lorenz()
prob = ODEProblem(lorenz,u0,tspan)  # Works
sol = solve(prob)
println("Completed problem 1")

# Try using lorenz!()
prob = ODEProblem(lorenz!,u0,tspan) # Exception
sol = solve(prob)
println("Completed problem 2")

Details - This code solves example 2 from the documentation using both an RHS that allocates lorenz() and another that uses in-place updating lorenz!().

The in-place updating version fails when run with the debugger in VS Code with the following error

Exception has occurred: ErrorException
`llvmcall` must be compiled to be called

Stacktrace:
  [1] assume(v::Bool)
    @ FunctionWrappers ~/.julia/packages/FunctionWrappers/8xdVB/src/FunctionWrappers.jl:10
  [2] do_ccall(f::FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{Float64}, Vector{Float64}, SciMLBase.NullParameters, Float64}}, args::Tuple{Vector{Float64}, Vector{Float64}, SciMLBase.NullParameters, Float64})
    @ FunctionWrappers ~/.julia/packages/FunctionWrappers/8xdVB/src/FunctionWrappers.jl:128
  [3] (::FunctionWrappers.FunctionWrapper{Nothing, Tuple{Vector{Float64}, Vector{Float64}, SciMLBase.NullParameters, Float64}})(args::Tuple{Vector{Float64}, Vector{Float64}, SciMLBase.NullParameters, Float64})
    @ FunctionWrappers ~/.julia/packages/FunctionWrappers/8xdVB/src/FunctionWrappers.jl:137

[see attached file for full stacktrace]

error.txt VerisonInfo.txt

[ChrisRackauckas]

That's interesting. This is a standard binary from https://julialang.org/downloads/ ?

[markowkes]

Yes, it's macOS ARM (M-series Processor) version 1.8.1. I see that 1.8.2 has now been released.

---

Mark Owkes, Ph.D. | he/him/his | Associate Professor 406.994.6300 | www.montana.edu/mowkeshttp://www.montana.edu/mowkes | @.**@.> Montana State University | Mechanical & Industrial Engineering Office: 201 Roberts Hall | P.O. Box 173800, Bozeman, MT 59717-3800

On Sep 30, 2022, 4:21 PM -0400, Christopher Rackauckas @.***>, wrote:

External Sender

That's interesting. This is a standard binary from https://julialang.org/downloads/https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjulialang.org%2Fdownloads%2F&data=05%7C01%7Cmark.owkes%40montana.edu%7C282a2f44ee5d4722825a08daa32153ac%7C324aa97a03a644fc91e43846fbced113%7C0%7C0%7C638001660795311700%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=j1gEEDuSJLIoeQLbn%2F7r1m%2BylTl4jIeCPTlmSns9cVU%3D&reserved=0 ?

— Reply to this email directly, view it on GitHubhttps://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSciML%2FDifferentialEquations.jl%2Fissues%2F905%23issuecomment-1263987160&data=05%7C01%7Cmark.owkes%40montana.edu%7C282a2f44ee5d4722825a08daa32153ac%7C324aa97a03a644fc91e43846fbced113%7C0%7C0%7C638001660795311700%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=txaHPm0Ypq3UnLBari9Dmv5Og5m2xkno%2FXeCuBGSwiA%3D&reserved=0, or unsubscribehttps://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEC6XOEUZM6LIPOSRPEGYFTWA5DTVANCNFSM6AAAAAAQZ3CDCY&data=05%7C01%7Cmark.owkes%40montana.edu%7C282a2f44ee5d4722825a08daa32153ac%7C324aa97a03a644fc91e43846fbced113%7C0%7C0%7C638001660795311700%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=j6EUa3ynYIUCOr1RCfFKhDVUVmFWgTiy3sULPB3AtwU%3D&reserved=0. You are receiving this because you authored the thread.Message ID: @.***>

[ChrisRackauckas]

Okay interesting, I would've only expected this kind of error if someone was using an incorrect Linux distro build, like apt-get julia. If you're using the binary from exactly those links, it shouldn't be happening. @chriselrod you have an M1 right? Can you confirm?

As a workaround, try prob = ODEProblem{true, SciMLBase.FullSpecialize}(lorenz!,u0,tspan).

[gbaraldi]

At least on 1.8.2 on an empty environment it works fine for me. Could you try on 1.8.2/an empty environment ?

[chriselrod]

The problem was that it was run in a debugger.

Running the following code through the debugger in VS Code

I haven't tried that, but it probably fails on all architectures and OSes.

Outside of a debugger, it runs just fine (and very quickly) on my M1.

[ChrisRackauckas]

Okay. Well, for the debugger just use the FullSpecialize form then. Other than that, it sounds like this is a debugger issue.

[ChrisRackauckas]

I think we can solve this in a nicer way if we can say "and if not in the debugging environment" in some way without depending on the debugger. It would just go right here:

https://github.com/SciML/DiffEqBase.jl/blob/master/src/solve.jl#L978

@pfitzseb how would we do this?

[ChrisRackauckas]

Added to the FAQ for now: https://github.com/SciML/DiffEqDocs.jl/blob/master/docs/src/basics/faq.md#error-exception-llvmcall-must-be-compiled-to-be-called-when-running-the-debugger

[pfitzseb]

This should just be fixed in JuliaInterpreter instead of downstream having to work around it. Does this reproduce on non-ARM archs?

[ChrisRackauckas]

If JuliaInterpreter doesn't have a solution in a week, then we should work around it. Is there a solution coming?

[chriselrod]

I cannot reproduce simple llvmcall errors on ARM or x64 when using @interpret.

[Ge0rges]

Just wanted to chime in to say that I have gotten this error on M1, v1.8.2 in VSCode debugger. Julia installed via home-brew.

[ChrisRackauckas]

Well Homebrew is a broken installation IIRC. Use the binaries from https://julialang.org/ and see if it still happens.

[gbaraldi]

Homebrew is actually quite good. They ship with our patches etc. But testing with the official binaries might be helpful