search

EnzymeAD / Enzyme.jl

Julia bindings for the Enzyme automatic differentiator
https://enzyme.mit.edu
MIT License
439 stars 62 forks source link

`foldl` with `CUDA.jl` results in `jl_lazy_load_and_lookup` #272

Closed michel2323 closed 2 years ago

michel2323 commented 2 years ago

Not sure this deserves a new issue. Related to https://github.com/EnzymeAD/Enzyme.jl/issues/144 and https://github.com/EnzymeAD/Enzyme.jl/issues/265 , I'm trying to get reduce to work with CUDA.jl. reduce calls foldl AFAIK, where I was able to generate second-order derivatives. However, the gradient computation breaks using the master branch of CUDA.jl.

MWE using Julia 1.7.2, Enzyme 0.9.2, CUDA.jl master. log.txt

using Enzyme
using CUDA

n = 10
function speelpenning(y, x)
    CUDA.@allowscalar y .= foldl(*, x; init=1.0)
    return nothing
end

function reverse(y::VT, x::VT) where {VT}
    FT = eltype(x)
    rx = convert(VT, zeros(FT,n))
    ry = convert(VT, ones(FT,1))

    _x = Duplicated(x, rx)
    _y = Duplicated(y, ry)

    autodiff(speelpenning, _y, _x)
    return rx
end

x = [i/(1.0+i) for i in 1:n]
y = zeros(1)
speelpenning(y,x)

# g = reverse(y, x)

using CUDA

cux = x |> CuArray
cuy = y |> CuArray

speelpenning(cuy,cux)

cug = reverse(cuy, cux)

In https://github.com/EnzymeAD/Enzyme.jl/issues/144 it is mentioned that jl_lazy_load_and_lookup might be related to heterogeneous code, yet I don't see that here.

wsmoses commented 2 years ago

For sake of keeping a log, using regular autodiff results in the following error:

julia> cug = reverse(cuy, cux)
ERROR: InvalidIRError: compiling function #speelpenning(CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}, CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}) resulted in invalid LLVM IR
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:41
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuDeviceGetName
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] name
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:86
 [5] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:132
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuCtxPopCurrent_v2
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] pop!
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/context.jl:171
 [5] device
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/context.jl:288
 [6] check_exceptions
   @ ~/.julia/packages/CUDA/DfvRa/src/compiler/exceptions.jl:33
 [7] #synchronize#12
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/stream.jl:134
 [8] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] unsafe_cuCtxGetDevice
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:150
 [2] current_device
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:23
 [3] device
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/context.jl:287
 [4] check_exceptions
   @ ~/.julia/packages/CUDA/DfvRa/src/compiler/exceptions.jl:33
 [5] #synchronize#12
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/stream.jl:134
 [6] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:126
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuCtxPushCurrent_v2
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] push!
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/context.jl:162
 [5] device
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/context.jl:286
 [6] check_exceptions
   @ ~/.julia/packages/CUDA/DfvRa/src/compiler/exceptions.jl:33
 [7] #synchronize#12
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/stream.jl:134
 [8] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:29
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuDeviceGet
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] CuDevice
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:17
 [5] iterate
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:140
 [6] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:35
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuDeviceGetCount
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] ndevices
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:159
 [5] length
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:143
 [6] iterate
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:140
 [7] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:23
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuDriverGetVersion
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] version
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/version.jl:24
 [5] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:59
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuDeviceGetAttribute
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] attribute
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:175
 [5] memory_pools_supported
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:198
 [6] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/src/pool.jl:72
 [7] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/memoization.jl:78
 [8] stream_ordered
   @ ~/.julia/packages/CUDA/DfvRa/src/pool.jl:71
 [9] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:394
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuMemHostAlloc
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] alloc
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/memory.jl:160
 [5] create_exceptions!
   @ ~/.julia/packages/CUDA/DfvRa/src/compiler/exceptions.jl:20
 [6] #cufunction#221
   @ ~/.julia/packages/CUDA/DfvRa/src/compiler/execution.jl:308
 [7] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [3] cuCtxGetCurrent
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] prepare_cuda_state
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:94
  [5] initialize_context
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:80
  [6] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:393
  [7] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [8] cuMemHostAlloc
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [9] alloc
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/memory.jl:160
 [10] create_exceptions!
    @ ~/.julia/packages/CUDA/DfvRa/src/compiler/exceptions.jl:20
 [11] #cufunction#221
    @ ~/.julia/packages/CUDA/DfvRa/src/compiler/execution.jl:308
 [12] multiple call sites
    @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:934
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuStreamCreate
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] CuStream
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/stream.jl:20
 [5] CuStream
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/stream.jl:18
 [6] create_stream
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:354
 [7] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [3] cuCtxGetCurrent
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] prepare_cuda_state
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:94
  [5] initialize_context
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:80
  [6] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:933
  [7] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [8] cuStreamCreate
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [9] CuStream
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/stream.jl:20
 [10] CuStream
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/stream.jl:18
 [11] create_stream
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:354
 [12] multiple call sites
    @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuCtxGetCurrent
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] current_context
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/context.jl:63
 [5] CuStream
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/stream.jl:26
 [6] CuStream
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/stream.jl:18
 [7] create_stream
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:354
 [8] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] nvtxNameCuStreamA
   @ ~/.julia/packages/CUDA/DfvRa/lib/nvtx/libnvtx.jl:244
 [2] create_stream
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:360
 [3] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:2277
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuDeviceGetMemPool
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] memory_pool
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/pool.jl:46
 [5] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [3] cuCtxGetCurrent
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] prepare_cuda_state
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:94
  [5] initialize_context
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:80
  [6] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:2276
  [7] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [8] cuDeviceGetMemPool
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [9] memory_pool
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/pool.jl:46
 [10] multiple call sites
    @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuCtxGetCurrent
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] current_context
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/context.jl:63
 [5] memory_pool
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/pool.jl:48
 [6] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:35
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [3] cuDeviceGetCount
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] ndevices
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:159
  [5] #202
    @ ~/.julia/packages/CUDA/DfvRa/src/compiler/gpucompiler.jl:11
  [6] initialize!
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/threading.jl:36
  [7] #get!#6
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/threading.jl:26
  [8] get!
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/threading.jl:25
  [9] device_properties
    @ ~/.julia/packages/CUDA/DfvRa/src/compiler/gpucompiler.jl:7
 [10] #CUDACompilerTarget#206
    @ ~/.julia/packages/CUDA/DfvRa/src/compiler/gpucompiler.jl:51
 [11] CUDACompilerTarget
    @ ~/.julia/packages/CUDA/DfvRa/src/compiler/gpucompiler.jl:51
 [12] #cufunction#221
    @ ~/.julia/packages/CUDA/DfvRa/src/compiler/execution.jl:296
 [13] multiple call sites
    @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:138
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuCtxSetCurrent
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] activate
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/context.jl:181
 [5] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] unsafe_cuCtxGetDevice
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:150
 [2] current_device
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:23
 [3] context!
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:150
 [4] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:138
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuCtxSetCurrent
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] activate
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/context.jl:181
 [5] context!
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:149
 [6] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] unsafe_cuCtxGetDevice
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:150
 [2] current_device
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:23
 [3] context!
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:144
 [4] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:138
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuCtxSetCurrent
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] activate
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/context.jl:181
 [5] context!
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:143
 [6] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] unsafe_cuPointerGetAttribute
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:883
 [2] is_pinned
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/memory.jl:849
 [3] #185
   @ ~/.julia/packages/CUDA/DfvRa/src/array.jl:410
 [4] #context!#63
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:164
 [5] context!
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:161
 [6] unsafe_copyto!
   @ ~/.julia/packages/CUDA/DfvRa/src/array.jl:406
 [7] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [3] cuCtxGetCurrent
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] prepare_cuda_state
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:94
  [5] initialize_context
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:80
  [6] unsafe_cuPointerGetAttribute
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:882
  [7] is_pinned
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/memory.jl:849
  [8] #185
    @ ~/.julia/packages/CUDA/DfvRa/src/array.jl:410
  [9] #context!#63
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:164
 [10] context!
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:161
 [11] unsafe_copyto!
    @ ~/.julia/packages/CUDA/DfvRa/src/array.jl:406
 [12] multiple call sites
    @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] unsafe_cuStreamQuery
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:1025
 [2] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuCtxGetCurrent
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] prepare_cuda_state
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:94
 [5] initialize_context
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:80
 [6] unsafe_cuStreamQuery
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:1024
 [7] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:401
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuMemHostGetDevicePointer_v2
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] convert
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/memory.jl:132
 [5] create_exceptions!
   @ ~/.julia/packages/CUDA/DfvRa/src/compiler/exceptions.jl:22
 [6] #cufunction#221
   @ ~/.julia/packages/CUDA/DfvRa/src/compiler/execution.jl:308
 [7] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [3] cuCtxGetCurrent
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] prepare_cuda_state
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:94
  [5] initialize_context
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:80
  [6] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:400
  [7] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [8] cuMemHostGetDevicePointer_v2
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [9] convert
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/memory.jl:132
 [10] create_exceptions!
    @ ~/.julia/packages/CUDA/DfvRa/src/compiler/exceptions.jl:22
 [11] #cufunction#221
    @ ~/.julia/packages/CUDA/DfvRa/src/compiler/execution.jl:308
 [12] multiple call sites
    @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:1634
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuOccupancyMaxPotentialBlockSize
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuCtxGetCurrent
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] prepare_cuda_state
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:94
 [5] initialize_context
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:80
 [6] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:1633
 [7] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [8] cuOccupancyMaxPotentialBlockSize
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [9] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:1254
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuLaunchHostFunc
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] #launch#46
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/execution.jl:159
 [5] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [3] cuCtxGetCurrent
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] prepare_cuda_state
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:94
  [5] initialize_context
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:80
  [6] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:1253
  [7] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [8] cuLaunchHostFunc
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [9] #launch#46
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/execution.jl:159
 [10] multiple call sites
    @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:35
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [3] cuDeviceGetCount
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] ndevices
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:159
  [5] #64
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:190
  [6] initialize!
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/threading.jl:36
  [7] #get!#6
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/threading.jl:26
  [8] get!
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/threading.jl:25
  [9] device_contexts
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:189
 [10] multiple call sites
    @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:59
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuDeviceGetAttribute
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] attribute
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:175
 [5] capability
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:194
 [6] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:84
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuDevicePrimaryCtxRetain
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] CuContext
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/context.jl:57
 [5] context
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:222
 [6] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:35
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuDeviceGetCount
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] ndevices
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:159
 [5] TaskLocalState
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:53
 [6] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:29
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuDeviceGet
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] CuDevice
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:17
 [5] TaskLocalState
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:50
 [6] task_local_state!
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:73
 [7] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:345
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuMemGetInfo_v2
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] info
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/memory.jl:754
 [5] MemoryInfo
   @ ~/.julia/packages/CUDA/DfvRa/src/pool.jl:154
 [6] OutOfGPUMemoryError (repeats 2 times)
   @ ~/.julia/packages/CUDA/DfvRa/src/pool.jl:207
 [7] throw_api_error
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:87
 [8] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [3] cuCtxGetCurrent
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] prepare_cuda_state
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:94
  [5] initialize_context
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:80
  [6] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:344
  [7] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [8] cuMemGetInfo_v2
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [9] info
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/memory.jl:754
 [10] MemoryInfo
    @ ~/.julia/packages/CUDA/DfvRa/src/pool.jl:154
 [11] OutOfGPUMemoryError (repeats 2 times)
    @ ~/.julia/packages/CUDA/DfvRa/src/pool.jl:207
 [12] throw_api_error
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:87
 [13] multiple call sites
    @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:35
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuDeviceGetCount
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] ndevices
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/devices.jl:159
 [5] #157
   @ ./none:0
 [6] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:598
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuMemcpyDtoHAsync_v2
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] #unsafe_copyto!#8
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/memory.jl:397
 [5] #185
   @ ~/.julia/packages/CUDA/DfvRa/src/array.jl:413
 [6] #context!#63
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:164
 [7] context!
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:161
 [8] unsafe_copyto!
   @ ~/.julia/packages/CUDA/DfvRa/src/array.jl:406
 [9] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [3] cuCtxGetCurrent
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] prepare_cuda_state
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:94
  [5] initialize_context
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:80
  [6] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:597
  [7] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [8] cuMemcpyDtoHAsync_v2
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [9] #unsafe_copyto!#8
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/memory.jl:397
 [10] #185
    @ ~/.julia/packages/CUDA/DfvRa/src/array.jl:413
 [11] #context!#63
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:164
 [12] context!
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:161
 [13] unsafe_copyto!
    @ ~/.julia/packages/CUDA/DfvRa/src/array.jl:406
 [14] multiple call sites
    @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:1032
 [2] macro expansion
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
 [3] cuStreamSynchronize
   @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
 [4] #synchronize#12
   @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/stream.jl:132
 [5] multiple call sites
   @ unknown:0
Reason: unsupported jl_lazy_load_and_lookup
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:144
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [3] cuCtxGetCurrent
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] prepare_cuda_state
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/state.jl:94
  [5] initialize_context
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:80
  [6] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/libcuda.jl:1031
  [7] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/error.jl:95
  [8] cuStreamSynchronize
    @ ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [9] #synchronize#12
    @ ~/.julia/packages/CUDA/DfvRa/lib/cudadrv/stream.jl:132
 [10] multiple call sites
    @ unknown:0
Hint: catch this exception as `err` and call `code_typed(err; interactive = true)` to introspect the erronous code
Stacktrace:
  [1] check_ir
    @ /mnt/Data/git/Enzyme.jl/src/compiler/validation.jl:135 [inlined]
  [2] codegen(output::Symbol, job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams, GPUCompiler.FunctionSpec{typeof(speelpenning), Tuple{CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}, CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}}}}; libraries::Bool, deferred_codegen::Bool, optimize::Bool, ctx::LLVM.Context, strip::Bool, validate::Bool, only_entry::Bool, parent_job::Nothing)
    @ Enzyme.Compiler /mnt/Data/git/Enzyme.jl/src/compiler.jl:3916
  [3] _thunk(job::GPUCompiler.CompilerJob{Enzyme.Compiler.EnzymeTarget, Enzyme.Compiler.EnzymeCompilerParams, GPUCompiler.FunctionSpec{typeof(speelpenning), Tuple{CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}, CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}}}})
    @ Enzyme.Compiler /mnt/Data/git/Enzyme.jl/src/compiler.jl:4614
  [4] cached_compilation(job::GPUCompiler.CompilerJob, key::UInt64, specid::UInt64)
    @ Enzyme.Compiler /mnt/Data/git/Enzyme.jl/src/compiler.jl:4652
  [5] #s566#115
    @ /mnt/Data/git/Enzyme.jl/src/compiler.jl:4712 [inlined]
  [6] var"#s566#115"(F::Any, Fn::Any, DF::Any, A::Any, TT::Any, Mode::Any, ModifiedBetween::Any, width::Any, specid::Any, ReturnPrimal::Any, ::Any, #unused#::Type, f::Any, df::Any, #unused#::Type, tt::Any, #unused#::Type, #unused#::Type, #unused#::Type, #unused#::Type, #unused#::Any)
    @ Enzyme.Compiler ./none:0
  [7] (::Core.GeneratedFunctionStub)(::Any, ::Vararg{Any})
    @ Core ./boot.jl:580
  [8] thunk
    @ /mnt/Data/git/Enzyme.jl/src/compiler.jl:4740 [inlined]
  [9] thunk (repeats 2 times)
    @ /mnt/Data/git/Enzyme.jl/src/compiler.jl:4733 [inlined]
 [10] autodiff
    @ /mnt/Data/git/Enzyme.jl/src/Enzyme.jl:285 [inlined]
 [11] autodiff
    @ /mnt/Data/git/Enzyme.jl/src/Enzyme.jl:319 [inlined]
 [12] autodiff
    @ /mnt/Data/git/Enzyme.jl/src/Enzyme.jl:412 [inlined]
 [13] reverse(y::CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}, x::CuArray{Float64, 1, CUDA.Mem.DeviceBuffer})
    @ Main ./REPL[6]:9
 [14] top-level scope
    @ REPL[14]:1
 [15] top-level scope
    @ ~/.julia/packages/CUDA/DfvRa/src/initialization.jl:52
vchuravy commented 2 years ago

Closing this is a duplicate of #144