JuliaGPU / CuArrays.jl

A Curious Cumulation of CUDA Cuisine
https://juliagpu.org/cuda/
Other
278 stars 83 forks source link

CuArrays test fail: an illegal memory access #342

Closed peraponf closed 5 years ago

peraponf commented 5 years ago

Describe the bug CuArrays test failed on "illegal memory access". It produced very long output, starting after test running CUBLAS. The symptom is similar to Issue #257. Update Julia to 1.1 the issue still persists.

Test Summary

Test Summary:                                                  | Pass  Error  Total
CuArrays                                                       | 1450    653   2103
  GPUArrays test suite                                         | 1023          1023
  Memory                                                       |    5             5
  Array                                                        |   19            19
  Adapt                                                        |    2             2
  Broadcast                                                    |   10            10
  Cufunc                                                       |    6             6
  Ref Broadcast                                                |    1             1
  Broadcast Fix                                                |    4             4
  Reduce                                                       |    6             6
  0D                                                           |    2             2
  Slices                                                       |   17            17
  Reshape                                                      |    1             1
  LinearAlgebra.triu! with diagonal -2                         |    1             1
  LinearAlgebra.triu! with diagonal -1                         |    1             1
  LinearAlgebra.triu! with diagonal 0                          |    1             1
  LinearAlgebra.triu! with diagonal 1                          |    1             1
  LinearAlgebra.triu! with diagonal 2                          |    1             1
  LinearAlgebra.tril! with diagonal -2                         |    1             1
  LinearAlgebra.tril! with diagonal -1                         |    1             1
  LinearAlgebra.tril! with diagonal 0                          |    1             1
  LinearAlgebra.tril! with diagonal 1                          |    1             1
  LinearAlgebra.tril! with diagonal 2                          |    1             1
  Utilities                                                    |    2             2
  accumulate                                                   |    8             8
  logical indexing                                             |   15            15
  CUDNN                                                        |   41            41
  CUBLAS                                                       |  262    162    424
    Level 1 with element type Float32                          |   10            10
    Level 1 with element type Float64                          |   10            10
    Level 1 with element type Complex{Float32}                 |    8             8
    Level 1 with element type Complex{Float64}                 |    8             8
    Level 2                                                    |  116           116
    Level 3                                                    |  102     74    176 
...
  CUSPARSE                                                     |   15    367    382
    util                                                       |           1      1
    char                                                       |   15            15
    conversion                                                 |          56     56
 CUSOLVER                                                     |    1     64     65
ERROR: LoadError: Some tests did not pass: 1450 passed, 0 failed, 653 errored, 0 broken.
in expression starting at C:\Users\P.Fakcharoenphol\.julia\packages\CuArrays\qZCAt\test\runtests.jl:25
error in running finalizer: CUDAdrv.CuError(code=700, meta=nothing)
error in running finalizer: CUDAdrv.CuError(code=700, meta=nothing)
...
error in running finalizer: CUDAdrv.CuError(code=700, meta=nothing)

Environment details

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = "C:\Users\P.Fakcharoenphol\AppData\Local\atom\app-1.36.1\atom.exe"  -a
  JULIA_NUM_THREADS = 4

CUDA: toolkit and driver version CUDA 10.0

diegozea commented 5 years ago

I've found a similar problem:

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

CUDA: 10.1.152 CuArrays tests needed cudnn and cuBLAS.

Test Summary:                                  | Pass  Error  Total
CuArrays                                       | 1322    549   1871
  GPUArrays test suite                         | 1025          1025
  Memory                                       |    5             5
  Array                                        |   22            22
  Adapt                                        |    2             2
  Broadcast                                    |   10            10
  Cufunc                                       |    6             6
  Ref Broadcast                                |    1             1
  Broadcast Fix                                |    4             4
  Reduce                                       |    6             6
  0D                                           |    2             2
  Slices                                       |   17            17
  Reshape                                      |    1             1
  LinearAlgebra.triu! with diagonal -2         |    1             1
  LinearAlgebra.triu! with diagonal -1         |    1             1
  LinearAlgebra.triu! with diagonal 0          |    1             1
  LinearAlgebra.triu! with diagonal 1          |    1             1
  LinearAlgebra.triu! with diagonal 2          |    1             1
  LinearAlgebra.tril! with diagonal -2         |    1             1
  LinearAlgebra.tril! with diagonal -1         |    1             1
  LinearAlgebra.tril! with diagonal 0          |    1             1
  LinearAlgebra.tril! with diagonal 1          |    1             1
  LinearAlgebra.tril! with diagonal 2          |    1             1
  Utilities                                    |    2             2
  accumulate                                   |    8             8
  logical indexing                             |   15            15
  generic fallbacks                            |   11            11
  CUDNN                                        |   70            70
  CUBLAS                                       |   89     58    147
    Level 1 with element type Float32          |   10            10
    Level 1 with element type Float64          |   10            10
    Level 1 with element type Complex{Float32} |    8             8
    Level 1 with element type Complex{Float64} |    8             8
    element type Float32                       |   49     13     62
      Level 2                                  |   23            23
      Level 3                                  |   26      4     30
        gemm!                                  |    2             2
        gemm                                   |    2             2
        xt_gemm!                               |    2             2
        xt_gemm                                |           1      1
        gemm_batched!                          |   10            10
        gemm_batched                           |   10            10
        gemm_strided_batched!                  |           1      1
        gemm_strided_batched                   |           1      1
      extensions                               |           9      9
        getrf_batched!                         |           1      1
        getrf_batched                          |           1      1
        getri_batched                          |           1      1
        matinv_batched                         |           1      1
        geqrf_batched!                         |           1      1
        geqrf_batched                          |           1      1
        gels_batched!                          |           1      1
        gels_batched                           |           1      1
    element type Float64                       |          15     15
      Level 2                                  |           5      5
        gemv                                   |           3      3
        banded methods                         |           1      1
      Level 3                                  |           1      1
      extensions                               |           9      9
        getrf_batched!                         |           1      1
        getrf_batched                          |           1      1
        getri_batched                          |           1      1
        matinv_batched                         |           1      1
        geqrf_batched!                         |           1      1
        geqrf_batched                          |           1      1
        gels_batched!                          |           1      1
        gels_batched                           |           1      1
    element type Complex{Float32}              |          15     15
      Level 2                                  |           5      5
        gemv                                   |           3      3
        banded methods                         |           1      1
      Level 3                                  |           1      1
      extensions                               |           9      9
        getrf_batched!                         |           1      1
        getrf_batched                          |           1      1
        getri_batched                          |           1      1
        matinv_batched                         |           1      1
        geqrf_batched!                         |           1      1
        geqrf_batched                          |           1      1
        gels_batched!                          |           1      1
        gels_batched                           |           1      1
    element type Complex{Float64}              |          15     15
      Level 2                                  |           5      5
        gemv                                   |           3      3
        banded methods                         |           1      1
      Level 3                                  |           1      1
      extensions                               |           9      9
        getrf_batched!                         |           1      1
        getrf_batched                          |           1      1
        getri_batched                          |           1      1
        matinv_batched                         |           1      1
        geqrf_batched!                         |           1      1
        geqrf_batched                          |           1      1
        gels_batched!                          |           1      1
        gels_batched                           |           1      1
  CUSPARSE                                     |   15    367    382
    util                                       |           1      1
    char                                       |   15            15
    conversion                                 |          56     56
      elty = Float32                           |          14     14
        make_csc                               |           1      1
        make_csr                               |           1      1
        convert_r2c                            |           1      1
        convert_r2b                            |           1      1
        convert_c2b                            |           1      1
        convert_c2h                            |           1      1
        convert_r2h                            |           1      1
        convert_d2h                            |           1      1
        convert_d2b                            |           1      1
        convert_c2r                            |           1      1
        convert_r2d                            |           1      1
        convert_c2d                            |           1      1
        convert_d2c                            |           1      1
        convert_d2r                            |           1      1
      elty = Float64                           |          14     14
        make_csc                               |           1      1
        make_csr                               |           1      1
        convert_r2c                            |           1      1
        convert_r2b                            |           1      1
        convert_c2b                            |           1      1
        convert_c2h                            |           1      1
        convert_r2h                            |           1      1
        convert_d2h                            |           1      1
        convert_d2b                            |           1      1
        convert_c2r                            |           1      1
        convert_r2d                            |           1      1
        convert_c2d                            |           1      1
        convert_d2c                            |           1      1
        convert_d2r                            |           1      1
      elty = Complex{Float32}                  |          14     14
        make_csc                               |           1      1
        make_csr                               |           1      1
        convert_r2c                            |           1      1
        convert_r2b                            |           1      1
        convert_c2b                            |           1      1
        convert_c2h                            |           1      1
        convert_r2h                            |           1      1
        convert_d2h                            |           1      1
        convert_d2b                            |           1      1
        convert_c2r                            |           1      1
        convert_r2d                            |           1      1
        convert_c2d                            |           1      1
        convert_d2c                            |           1      1
        convert_d2r                            |           1      1
      elty = Complex{Float64}                  |          14     14
        make_csc                               |           1      1
        make_csr                               |           1      1
        convert_r2c                            |           1      1
        convert_r2b                            |           1      1
        convert_c2b                            |           1      1
        convert_c2h                            |           1      1
        convert_r2h                            |           1      1
        convert_d2h                            |           1      1
        convert_d2b                            |           1      1
        convert_c2r                            |           1      1
        convert_r2d                            |           1      1
        convert_c2d                            |           1      1
        convert_d2c                            |           1      1
        convert_d2r                            |           1      1
    bsric02                                    |           8      8
      elty = Float32                           |           2      2
        bsric02!                               |           1      1
        bsric02                                |           1      1
      elty = Float64                           |           2      2
        bsric02!                               |           1      1
        bsric02                                |           1      1
      elty = Complex{Float32}                  |           2      2
        bsric02!                               |           1      1
        bsric02                                |           1      1
      elty = Complex{Float64}                  |           2      2
        bsric02!                               |           1      1
        bsric02                                |           1      1
    bsrilu02                                   |           8      8
      elty = Float32                           |           2      2
        bsrilu02!                              |           1      1
        bsrilu02                               |           1      1
      elty = Float64                           |           2      2
        bsrilu02!                              |           1      1
        bsrilu02                               |           1      1
      elty = Complex{Float32}                  |           2      2
        bsrilu02!                              |           1      1
        bsrilu02                               |           1      1
      elty = Complex{Float64}                  |           2      2
        bsrilu02!                              |           1      1
        bsrilu02                               |           1      1
    bsrsm2                                     |           8      8
      elty = Float32                           |           2      2
        bsrsm2!                                |           1      1
        bsrsm2                                 |           1      1
      elty = Float64                           |           2      2
        bsrsm2!                                |           1      1
        bsrsm2                                 |           1      1
      elty = Complex{Float32}                  |           2      2
        bsrsm2!                                |           1      1
        bsrsm2                                 |           1      1
      elty = Complex{Float64}                  |           2      2
        bsrsm2!                                |           1      1
        bsrsm2                                 |           1      1
    bsrsv2                                     |           8      8
      elty = Float32                           |           2      2
        bsrsv2!                                |           1      1
        bsrsv2                                 |           1      1
      elty = Float64                           |           2      2
        bsrsv2!                                |           1      1
        bsrsv2                                 |           1      1
      elty = Complex{Float32}                  |           2      2
        bsrsv2!                                |           1      1
        bsrsv2                                 |           1      1
      elty = Complex{Float64}                  |           2      2
        bsrsv2!                                |           1      1
        bsrsv2                                 |           1      1
    ilu0                                       |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    ilu02                                      |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    ic0                                        |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    ic2                                        |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    csrsm                                      |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    cssv                                       |          16     16
      elty = Float32                           |           4      4
        csrsv                                  |           1      1
        cscsv                                  |           1      1
        csrsv2                                 |           1      1
        cscsv2                                 |           1      1
      elty = Float64                           |           4      4
        csrsv                                  |           1      1
        cscsv                                  |           1      1
        csrsv2                                 |           1      1
        cscsv2                                 |           1      1
      elty = Complex{Float32}                  |           4      4
        csrsv                                  |           1      1
        cscsv                                  |           1      1
        csrsv2                                 |           1      1
        cscsv2                                 |           1      1
      elty = Complex{Float64}                  |           4      4
        csrsv                                  |           1      1
        cscsv                                  |           1      1
        csrsv2                                 |           1      1
        cscsv2                                 |           1      1
    doti                                       |           4      4
      elty = Float32                           |           1      1
      elty = Float64                           |           1      1
      elty = Complex{Float32}                  |           1      1
      elty = Complex{Float64}                  |           1      1
    dotci                                      |           2      2
      elty = Complex{Float32}                  |           1      1
      elty = Complex{Float64}                  |           1      1
    axpyi                                      |           8      8
      elty = Float32                           |           2      2
        axpyi!                                 |           1      1
        axpyi                                  |           1      1
      elty = Float64                           |           2      2
        axpyi!                                 |           1      1
        axpyi                                  |           1      1
      elty = Complex{Float32}                  |           2      2
        axpyi!                                 |           1      1
        axpyi                                  |           1      1
      elty = Complex{Float64}                  |           2      2
        axpyi!                                 |           1      1
        axpyi                                  |           1      1
    geam                                       |          12     12
      elty = Float32                           |           3      3
        csr                                    |           1      1
        csc                                    |           1      1
        mixed                                  |           1      1
      elty = Float64                           |           3      3
        csr                                    |           1      1
        csc                                    |           1      1
        mixed                                  |           1      1
      elty = Complex{Float32}                  |           3      3
        csr                                    |           1      1
        csc                                    |           1      1
        mixed                                  |           1      1
      elty = Complex{Float64}                  |           3      3
        csr                                    |           1      1
        csc                                    |           1      1
        mixed                                  |           1      1
    gemm                                       |           4      4
      elty = Float32                           |           1      1
      elty = Float64                           |           1      1
      elty = Complex{Float32}                  |           1      1
      elty = Complex{Float64}                  |           1      1
    gthr and gthrz                             |          16     16
      elty = Float32                           |           4      4
        gthr!                                  |           1      1
        gthr                                   |           1      1
        gthrz!                                 |           1      1
        gthrz                                  |           1      1
      elty = Float64                           |           4      4
        gthr!                                  |           1      1
        gthr                                   |           1      1
        gthrz!                                 |           1      1
        gthrz                                  |           1      1
      elty = Complex{Float32}                  |           4      4
        gthr!                                  |           1      1
        gthr                                   |           1      1
        gthrz!                                 |           1      1
        gthrz                                  |           1      1
      elty = Complex{Float64}                  |           4      4
        gthr!                                  |           1      1
        gthr                                   |           1      1
        gthrz!                                 |           1      1
        gthrz                                  |           1      1
    gtsv                                       |          24     24
      elty = Float32                           |           6      6
        gtsv!                                  |           1      1
        gtsv                                   |           1      1
        gtsv_nopivot!                          |           1      1
        gtsv_nopivot                           |           1      1
        gtsvStridedBatch!                      |           1      1
        gtsvStridedBatch                       |           1      1
      elty = Float64                           |           6      6
        gtsv!                                  |           1      1
        gtsv                                   |           1      1
        gtsv_nopivot!                          |           1      1
        gtsv_nopivot                           |           1      1
        gtsvStridedBatch!                      |           1      1
        gtsvStridedBatch                       |           1      1
      elty = Complex{Float32}                  |           6      6
        gtsv!                                  |           1      1
        gtsv                                   |           1      1
        gtsv_nopivot!                          |           1      1
        gtsv_nopivot                           |           1      1
        gtsvStridedBatch!                      |           1      1
        gtsvStridedBatch                       |           1      1
      elty = Complex{Float64}                  |           6      6
        gtsv!                                  |           1      1
        gtsv                                   |           1      1
        gtsv_nopivot!                          |           1      1
        gtsv_nopivot                           |           1      1
        gtsvStridedBatch!                      |           1      1
        gtsvStridedBatch                       |           1      1
    hybsv                                      |           4      4
      elty = Float32                           |           1      1
      elty = Float64                           |           1      1
      elty = Complex{Float32}                  |           1      1
      elty = Complex{Float64}                  |           1      1
    mm                                         |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    mm_symm                                    |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    mm_herm                                    |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    mm2                                        |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    bsrmm2                                     |           4      4
      elty = Float32                           |           1      1
      elty = Float64                           |           1      1
      elty = Complex{Float32}                  |           1      1
      elty = Complex{Float64}                  |           1      1
    mv                                         |          32     32
      mv_symm                                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv_herm                                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv                                       |           4      4
        csr                                    |           1      1
        csc                                    |           1      1
        bsr                                    |           1      1
        hyb                                    |           1      1
      mv_symm                                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv_herm                                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv                                       |           4      4
        csr                                    |           1      1
        csc                                    |           1      1
        bsr                                    |           1      1
        hyb                                    |           1      1
      mv_symm                                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv_herm                                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv                                       |           4      4
        csr                                    |           1      1
        csc                                    |           1      1
        bsr                                    |           1      1
        hyb                                    |           1      1
      mv_symm                                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv_herm                                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv                                       |           4      4
        csr                                    |           1      1
        csc                                    |           1      1
        bsr                                    |           1      1
        hyb                                    |           1      1
    mm!                                        |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    mm_symm!                                   |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    mm_herm!                                   |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    mm2!                                       |           8      8
      elty = Float32                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Float64                           |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float32}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      elty = Complex{Float64}                  |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
    bsrmm2!                                    |           4      4
      elty = Float32                           |           1      1
      elty = Float64                           |           1      1
      elty = Complex{Float32}                  |           1      1
      elty = Complex{Float64}                  |           1      1
    mv!                                        |          32     32
      mv_symm!                                 |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv_herm!                                 |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv!                                      |           4      4
        csr                                    |           1      1
        csc                                    |           1      1
        bsr                                    |           1      1
        hyb                                    |           1      1
      mv_symm!                                 |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv_herm!                                 |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv!                                      |           4      4
        csr                                    |           1      1
        csc                                    |           1      1
        bsr                                    |           1      1
        hyb                                    |           1      1
      mv_symm!                                 |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv_herm!                                 |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv!                                      |           4      4
        csr                                    |           1      1
        csc                                    |           1      1
        bsr                                    |           1      1
        hyb                                    |           1      1
      mv_symm!                                 |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv_herm!                                 |           2      2
        csr                                    |           1      1
        csc                                    |           1      1
      mv!                                      |           4      4
        csr                                    |           1      1
        csc                                    |           1      1
        bsr                                    |           1      1
        hyb                                    |           1      1
    sctr                                       |           8      8
      elty = Float32                           |           2      2
        sctr!                                  |           1      1
        sctr                                   |           1      1
      elty = Float64                           |           2      2
        sctr!                                  |           1      1
        sctr                                   |           1      1
      elty = Complex{Float32}                  |           2      2
        sctr!                                  |           1      1
        sctr                                   |           1      1
      elty = Complex{Float64}                  |           2      2
        sctr!                                  |           1      1
        sctr                                   |           1      1
    roti                                       |           4      4
      elty = Float32                           |           2      2
        roti!                                  |           1      1
        roti                                   |           1      1
      elty = Float64                           |           2      2
        roti!                                  |           1      1
        roti                                   |           1      1
  CUSOLVER                                     |    1     64     65
    elty = Float32                             |          16     16
      Cholesky (po)                            |           1      1
      getrf!                                   |           1      1
      getrs!                                   |           1      1
      geqrf!                                   |           1      1
      ormqr!                                   |           1      1
      orgqr!                                   |           1      1
      sytrf!                                   |           1      1
      gebrd!                                   |           1      1
      syevd!                                   |           1      1
      sygvd!                                   |           1      1
      syevj!                                   |           1      1
      svd with QRAlgorithm method              |           1      1
      svd with QRAlgorithm method              |           1      1
      svd with JacobiAlgorithm method          |           1      1
      svd with JacobiAlgorithm method          |           1      1
    elty = Float64                             |          16     16
      Cholesky (po)                            |           1      1
      getrf!                                   |           1      1
      getrs!                                   |           1      1
      geqrf!                                   |           1      1
      ormqr!                                   |           1      1
      orgqr!                                   |           1      1
      sytrf!                                   |           1      1
      gebrd!                                   |           1      1
      syevd!                                   |           1      1
      sygvd!                                   |           1      1
      syevj!                                   |           1      1
      svd with QRAlgorithm method              |           1      1
      svd with QRAlgorithm method              |           1      1
      svd with JacobiAlgorithm method          |           1      1
      svd with JacobiAlgorithm method          |           1      1
    elty = Complex{Float32}                    |          16     16
      Cholesky (po)                            |           1      1
      getrf!                                   |           1      1
      getrs!                                   |           1      1
      geqrf!                                   |           1      1
      ormqr!                                   |           1      1
      orgqr!                                   |           1      1
      sytrf!                                   |           1      1
      gebrd!                                   |           1      1
      syevd!                                   |           1      1
      sygvd!                                   |           1      1
      syevj!                                   |           1      1
      svd with QRAlgorithm method              |           1      1
      svd with QRAlgorithm method              |           1      1
      svd with JacobiAlgorithm method          |           1      1
      svd with JacobiAlgorithm method          |           1      1
    elty = Complex{Float64}                    |          16     16
      Cholesky (po)                            |           1      1
      getrf!                                   |           1      1
      getrs!                                   |           1      1
      geqrf!                                   |           1      1
      ormqr!                                   |           1      1
      orgqr!                                   |           1      1
      sytrf!                                   |           1      1
      gebrd!                                   |           1      1
      syevd!                                   |           1      1
      sygvd!                                   |           1      1
      syevj!                                   |           1      1
      svd with QRAlgorithm method              |           1      1
      svd with QRAlgorithm method              |           1      1
      svd with JacobiAlgorithm method          |           1      1
      svd with JacobiAlgorithm method          |           1      1
  CUFFT                                        |          34     34
    FFT                                        |          15     15
      1D FFT                                   |           1      1
      1D inplace FFT                           |           1      1
      2D FFT                                   |           1      1
      2D inplace FFT                           |           1      1
      Batch 1D FFT                             |           1      1
      3D FFT                                   |           1      1
      3D inplace FFT                           |           1      1
      Batch 2D FFT (in 3D)                     |           1      1
      Batch 2D FFT (in 4D)                     |           1      1
      1D real FFT                              |           1      1
      Batch 1D real FFT                        |           1      1
      2D real FFT                              |           1      1
      Batch 2D real FFT (in 3D)                |           1      1
      Batch 2D real FFT (in 4D)                |           1      1
      3D real FFT                              |           1      1
    FFT                                        |          15     15
      1D FFT                                   |           1      1
      1D inplace FFT                           |           1      1
      2D FFT                                   |           1      1
      2D inplace FFT                           |           1      1
      Batch 1D FFT                             |           1      1
      3D FFT                                   |           1      1
      3D inplace FFT                           |           1      1
      Batch 2D FFT (in 3D)                     |           1      1
      Batch 2D FFT (in 4D)                     |           1      1
      1D real FFT                              |           1      1
      Batch 1D real FFT                        |           1      1
      2D real FFT                              |           1      1
      Batch 2D real FFT (in 3D)                |           1      1
      Batch 2D real FFT (in 4D)                |           1      1
      3D real FFT                              |           1      1
    Int FFT                                    |           2      2
      1D FFT                                   |           1      1
      1D real FFT                              |           1      1
    Int FFT                                    |           2      2
      1D FFT                                   |           1      1
      1D real FFT                              |           1      1
  CURAND                                       |           1      1
  CUSPARSE + CUSOLVER                          |          24     24
    elty = Float32                             |           6      6
      csrlsvlu!                                |           1      1
      csrlsvqr!                                |           1      1
      csrlsvchol!                              |           1      1
      csreigvsi                                |           1      1
      csreigs                                  |           1      1
      csrlsqvqr!                               |           1      1
    elty = Float64                             |           6      6
      csrlsvlu!                                |           1      1
      csrlsvqr!                                |           1      1
      csrlsvchol!                              |           1      1
      csreigvsi                                |           1      1
      csreigs                                  |           1      1
      csrlsqvqr!                               |           1      1
    elty = Complex{Float32}                    |           6      6
      csrlsvlu!                                |           1      1
      csrlsvqr!                                |           1      1
      csrlsvchol!                              |           1      1
      csreigvsi                                |           1      1
      csreigs                                  |           1      1
      csrlsqvqr!                               |           1      1
    elty = Complex{Float64}                    |           6      6
      csrlsvlu!                                |           1      1
      csrlsvqr!                                |           1      1
      csrlsvchol!                              |           1      1
      csreigvsi                                |           1      1
      csreigs                                  |           1      1
      csrlsqvqr!                               |           1      1
ERROR: LoadError: Some tests did not pass: 1322 passed, 0 failed, 549 errored, 0 broken.
error in running finalizer: CUDAdrv.CuError(code=700, meta=nothing)
Batch 1D FFT: Error During Test at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\test\fft.jl:137
  Got exception outside of a @test
  CUDA error: an illegal memory access was encountered (code #700, ERROR_ILLEGAL_ADDRESS)
  Stacktrace:
   [1] alloc(::Type{CUDAdrv.Mem.DeviceBuffer}, ::Int64) at C:\Users\Diego\.julia\packages\CUDAdrv\3cR2F\src\base.jl:147
   [2] macro expansion at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\src\memory.jl:244 [inlined]
   [3] macro expansion at .\util.jl:213 [inlined]
   [4] try_cuda_alloc(::Int64) at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\src\memory.jl:243
   [5] macro expansion at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\src\memory.jl:262 [inlined]
   [6] macro expansion at C:\Users\Diego\.julia\packages\TimerOutputs\7zSea\src\TimerOutput.jl:216 [inlined]
   [7] try_alloc(::Int64, ::Int64) at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\src\memory.jl:261
   [8] macro expansion at C:\Users\Diego\.julia\packages\TimerOutputs\7zSea\src\TimerOutput.jl:216 [inlined]
   [9] (::getfield(CuArrays, Symbol("##15#16")))() at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\src\memory.jl:369
   [10] lock(::getfield(CuArrays, Symbol("##15#16")), ::ReentrantLock) at .\lock.jl:101
   [11] macro expansion at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\src\memory.jl:368 [inlined]
   [12] macro expansion at .\util.jl:213 [inlined]
   [13] alloc(::Int64) at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\src\memory.jl:358
   [14] CuArray{Complex{Float32},2}(::UndefInitializer, ::Tuple{Int64,Int64}) at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\src\array.jl:38
   [15] CuArray{Complex{Float32},2}(::Array{Complex{Float32},2}) at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\src\array.jl:106
   [16] Type at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\src\array.jl:116 [inlined]
   [17] (::getfield(Main, Symbol("#dotest3#168")))(::Array{Complex{Float32},2}, ::Int64) at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\test\fft.jl:86
   [18] top-level scope at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\test\fft.jl:140
   [19] top-level scope at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Test\src\Test.jl:1083
   [20] top-level scope at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\test\fft.jl:138
   [21] top-level scope at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Test\src\Test.jl:1156
   [22] top-level scope at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\test\fft.jl:113
   [23] top-level scope at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Test\src\Test.jl:1083
   [24] top-level scope at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\test\fft.jl:3
   [25] include at .\boot.jl:326 [inlined]
   [26] include_relative(::Module, ::String) at .\loading.jl:1038
   [27] include(::Module, ::String) at .\sysimg.jl:29
   [28] include(::String) at .\client.jl:403
   [29] top-level scope at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\test\runtests.jl:24
   [30] top-level scope at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Test\src\Test.jl:1083
   [31] top-level scope at C:\Users\Diego\.julia\packages\CuArrays\PwSdF\test\runtests.jl:19
   [32] include at .\boot.jl:326 [inlined]
   [33] include_relative(::Module, ::String) at .\loading.jl:1038
   [34] include(::Module, ::String) at .\sysimg.jl:29
   [35] include(::String) at .\client.jl:403
   [36] top-level scope at none:0
   [37] eval(::Module, ::Any) at .\boot.jl:328
   [38] exec_options(::Base.JLOptions) at .\client.jl:243
   [39] _start() at .\client.jl:436
peraponf commented 5 years ago

The problem is solved after:

  1. removed all CUDA 8.0 on my computer
  2. updated Julia to v. 1.1.1
  3. updated CUDA to v. 10.1.168
maleadt commented 5 years ago

Did that upgrade involve installing a new driver too? Tried reproducing on CUDA 8.0 myself, without success.

peraponf commented 5 years ago

Yes. I found my CUDA driver was v.8.0 after trying to rebuild CUDAdrv. This likely because when I was installing CUDA 10.0 it caused screen malfunction and I had to reinstall the driver from Labtop manufacturer website. I didn't realize it downgraded CUDA driver to v.8.0.

diegozea commented 5 years ago

I've installed Julia 1.1.1 and CUDA 10.1.168 but I'm still seeing the problems:

CUDA error: an illegal memory access was encountered (code #700, ERROR_ILLEGAL_ADDRESS)
CUSOLVERError(code 1, the library was not initialized)
error in running finalizer: CUDAdrv.CuError(code=700, meta=nothing)
peraponf commented 5 years ago

You may check if "C:\windows\SYSTEM32\nvcuda.DLL" is v.10.

diegozea commented 5 years ago

Yes, I have:

(Get-Item C:\windows\SYSTEM32\nvcuda.DLL).VersionInfo

ProductVersion   FileVersion      FileName
--------------   -----------      --------
25.21.14.2525    25.21.14.2525    C:\windows\SYSTEM32\nvcuda.DLL

image