FluxML / Flux.jl

Relax! Flux is the ML library that doesn't make you tensor
https://fluxml.ai/
Other
4.48k stars 603 forks source link

Pkg.test("Flux ") fails while Pkg.test("CuArrays") works on Julia 1.0.1 #646

Closed mleprovost closed 5 years ago

mleprovost commented 5 years ago

Hello, I am using Julia 1.0.1 on a GPU with CUDA 9.1. I would like to do GPU computing with Flux. But Flux fails at Pkg.test('Flux") while Pkg.test("CuArrays") works. Below are the outputs of Pkg.status(), Pkg.test("CuArrays") and Pkg.test("Flux")

Thank you for your help,

Result of Pkg.status():

using Pkg
Pkg.status()
    Status `~/.julia/environments/v1.0/Project.toml`
  [fbb218c0] BSON v0.2.1
  [6e4b80f9] BenchmarkTools v0.4.2
  [a134a8b2] BlackBoxOptim v0.4.0
  [336ed68f] CSV v0.4.3
  [5ae59095] Colors v0.9.5
  [150eb455] CoordinateTransformations v0.5.0+ #master (https://github.com/FugroRoames/CoordinateTransformations.jl.git)
  [3a865a2d] CuArrays v0.9.1
  [165a45c3] DASKR v2.4.0
  [e993076c] DASSL v2.3.0
  [a93c6f00] DataFrames v0.17.1
  [2b5f629d] DiffEqBase v5.4.0
  [aae7a2af] DiffEqFlux v0.2.0
  [1130ab10] DiffEqParamEstim v1.6.0
  [a077e3f3] DiffEqProblemLibrary v4.1.0
  [9cd584b6] DifferentialDynamicProgramming v0.3.1
  [0c46a032] DifferentialEquations v6.3.0
  [aaf54ef3] DistributedArrays v0.6.0
  [31c24e10] Distributions v0.16.4
  [442a2c76] FastGaussQuadrature v0.3.2
  [5789e2e9] FileIO v1.0.5
  [587475ba] Flux v0.7.3
  [f6369f11] ForwardDiff v0.10.3
  [28b8d3ca] GR v0.38.1
  [6f062d28] GeometricIntegrators v0.0.0 #master (https://github.com/DDMGNI/GeometricIntegrators.jl.git)
  [9fb69e20] Hiccup v0.2.2
  [7073ff75] IJulia v1.17.0
  [a98d9a8b] Interpolations v0.11.2
  [b6b21f68] Ipopt v0.5.4
  [4138dd39] JLD v0.9.1
  [033835bb] JLD2 v0.1.2
  [4076af6c] JuMP v0.19.0
  [e5e0dc1b] Juno v0.5.4
  [b964fa9f] LaTeXStrings v1.0.3
  [fdba3010] MathProgBase v0.7.7
  [76087f3c] NLopt v0.5.1
  [2774e3e8] NLsolve v3.0.1
  [54ca160b] ODEInterface v0.4.5
  [09606e27] ODEInterfaceDiffEq v3.0.0
  [429524aa] Optim v0.17.2
  [1dea7af3] OrdinaryDiffEq v5.3.0
  [65888b18] ParameterizedFunctions v4.1.1
  [2ae35dd2] Permutations v0.3.2
  [91a5bcdd] Plots v0.23.0
  [73af2aaf] PotentialFlow v0.1.0
  [4ce49e30] ProgressLogging v0.1.0 #master (https://github.com/adamslc/ProgressLogging.jl.git)
  [92933f4c] ProgressMeter v0.9.0
  [438e738f] PyCall v1.18.5
  [d330b81b] PyPlot v2.7.0
  [1fd47b50] QuadGK v2.0.3
  [731186ca] RecursiveArrayTools v0.20.0+ #master (https://github.com/JuliaDiffEq/RecursiveArrayTools.jl.git)
  [295af30f] Revise v1.0.2
  [c3572dad] Sundials v3.1.0
  [103da179] ViscousFlow v0.1.5 [`dev/ViscousFlow`]

Results of Pkg.test("CuArrays")

Pkg.test("CuArrays")
 Testing CuArrays
 Resolving package versions...
    Status `/tmp/tmpPeQMmJ/Manifest.toml`
  [621f4979] AbstractFFTs v0.3.2
  [79e6a3ab] Adapt v0.4.2
  [9e28174c] BinDeps v0.8.10
  [b99e7846] BinaryProvider v0.5.3
  [3895d2a7] CUDAapi v0.6.0
  [c5f51814] CUDAdrv v1.0.1
  [be33ccc6] CUDAnative v1.0.1
  [bbf7d656] CommonSubexpressions v0.2.0
  [34da2185] Compat v1.5.1
  [8f4d0f93] Conda v1.2.0
  [3a865a2d] CuArrays v0.9.1
  [163ba53b] DiffResults v0.0.4
  [b552c78f] DiffRules v0.0.10
  [7a1cc6ca] FFTW v0.2.4
  [1a297f60] FillArrays v0.4.0
  [f6369f11] ForwardDiff v0.10.3
  [0c68f7d7] GPUArrays v0.6.1
  [682c06a0] JSON v0.20.0
  [929cbde3] LLVM v1.0.0
  [1914dd2f] MacroTools v0.4.5
  [872c559c] NNlib v0.4.3
  [77ba4419] NaNMath v0.3.2
  [189a3867] Reexport v0.2.0
  [ae029012] Requires v0.5.2
  [276daf66] SpecialFunctions v0.7.2
  [90137ffa] StaticArrays v0.10.2
  [30578b45] URIParser v0.4.0
  [81def892] VersionParsing v1.1.3
  [2a0f44e3] Base64  [`@stdlib/Base64`]
  [ade2ca70] Dates  [`@stdlib/Dates`]
  [8bb1440f] DelimitedFiles  [`@stdlib/DelimitedFiles`]
  [8ba89e20] Distributed  [`@stdlib/Distributed`]
  [b77e0a4c] InteractiveUtils  [`@stdlib/InteractiveUtils`]
  [76f85450] LibGit2  [`@stdlib/LibGit2`]
  [8f399da3] Libdl  [`@stdlib/Libdl`]
  [37e2e46d] LinearAlgebra  [`@stdlib/LinearAlgebra`]
  [56ddb016] Logging  [`@stdlib/Logging`]
  [d6f4376e] Markdown  [`@stdlib/Markdown`]
  [a63ad114] Mmap  [`@stdlib/Mmap`]
  [44cfe95a] Pkg  [`@stdlib/Pkg`]
  [de0858da] Printf  [`@stdlib/Printf`]
  [3fa0cd96] REPL  [`@stdlib/REPL`]
  [9a3f8284] Random  [`@stdlib/Random`]
[ea8e919c] SHA  [`@stdlib/SHA`]
  [9e88b42a] Serialization  [`@stdlib/Serialization`]
  [1a1011a3] SharedArrays  [`@stdlib/SharedArrays`]
  [6462fe0b] Sockets  [`@stdlib/Sockets`]
  [2f01184e] SparseArrays  [`@stdlib/SparseArrays`]
  [10745b16] Statistics  [`@stdlib/Statistics`]
  [8dfed614] Test  [`@stdlib/Test`]
  [cf7118a7] UUIDs  [`@stdlib/UUIDs`]
  [4ec0a83e] Unicode  [`@stdlib/Unicode`]

[ Info: Testing CUDNN 7.1.2
[ Info: Testing CUBLAS 9.1.0
[ Info: Testing CUSPARSE 9.1.0
[ Info: Testing CUSOLVER 9.1.0
[ Info: Testing CUFFT 9.1.0
[ Info: Testing CURAND 9.1.0

Test Summary: | Pass  Total
CuArrays      | 3935   3935
   Testing CuArrays tests passed 

Results of Pkg.test("Flux"):

 Pkg.test("Flux")
   Testing Flux
    Status `/tmp/tmpy6v6zR/Manifest.toml`
  [1520ce14] AbstractTrees v0.2.1
  [79e6a3ab] Adapt v0.4.2
  [9e28174c] BinDeps v0.8.10
  [b99e7846] BinaryProvider v0.5.3
  [944b1d66] CodecZlib v0.5.1
  [3da002f7] ColorTypes v0.7.5
  [5ae59095] Colors v0.9.5
  [bbf7d656] CommonSubexpressions v0.2.0
  [34da2185] Compat v1.5.1
  [864edb3b] DataStructures v0.15.0
  [163ba53b] DiffResults v0.0.4
  [b552c78f] DiffRules v0.0.10
  [53c48c17] FixedPointNumbers v0.5.3
  [587475ba] Flux v0.7.3
  [f6369f11] ForwardDiff v0.10.3
  [e5e0dc1b] Juno v0.5.4
  [1914dd2f] MacroTools v0.4.5
  [e89f7d12] Media v0.5.0
  [e1d29d7a] Missings v0.4.0
  [872c559c] NNlib v0.4.3
  [77ba4419] NaNMath v0.3.2
  [bac558e1] OrderedCollections v1.0.2
  [189a3867] Reexport v0.2.0
  [ae029012] Requires v0.5.2
  [a2af1166] SortingAlgorithms v0.3.1
  [276daf66] SpecialFunctions v0.7.2
  [90137ffa] StaticArrays v0.10.2
  [2913bbd2] StatsBase v0.28.0
  [3bb67fe8] TranscodingStreams v0.8.1
  [30578b45] URIParser v0.4.0
  [a5390f91] ZipFile v0.8.0
  [2a0f44e3] Base64  [`@stdlib/Base64`]
  [ade2ca70] Dates  [`@stdlib/Dates`]
  [8bb1440f] DelimitedFiles  [`@stdlib/DelimitedFiles`]
  [8ba89e20] Distributed  [`@stdlib/Distributed`]
  [b77e0a4c] InteractiveUtils  [`@stdlib/InteractiveUtils`]
  [76f85450] LibGit2  [`@stdlib/LibGit2`]
  [8f399da3] Libdl  [`@stdlib/Libdl`]
  [37e2e46d] LinearAlgebra  [`@stdlib/LinearAlgebra`]
  [56ddb016] Logging  [`@stdlib/Logging`]
  [d6f4376e] Markdown  [`@stdlib/Markdown`]
  [a63ad114] Mmap  [`@stdlib/Mmap`]
  [44cfe95a] Pkg  [`@stdlib/Pkg`]
  [de0858da] Printf  [`@stdlib/Printf`]
  [9abbd945] Profile  [`@stdlib/Profile`]
  [3fa0cd96] REPL  [`@stdlib/REPL`]
  [9a3f8284] Random  [`@stdlib/Random`]
  [ea8e919c] SHA  [`@stdlib/SHA`]
  [9e88b42a] Serialization  [`@stdlib/Serialization`]
  [1a1011a3] SharedArrays  [`@stdlib/SharedArrays`]
  [6462fe0b] Sockets  [`@stdlib/Sockets`]
  [2f01184e] SparseArrays  [`@stdlib/SparseArrays`]
  [10745b16] Statistics  [`@stdlib/Statistics`]
  [8dfed614] Test  [`@stdlib/Test`]
  [cf7118a7] UUIDs  [`@stdlib/UUIDs`]
  [4ec0a83e] Unicode  [`@stdlib/Unicode`]
[ Info: Testing Basics
[ Info: Testing Layers
[ Info: Running Gradient Checks
[ Info: Testing GPU Support

onecold gpu: Error During Test at /u/home/m/mleprovo/.julia/packages/Flux/8XpDt/test/cuda/cuda.jl:41
  Got exception outside of a @test
  scalar setindex! is disallowed
  Stacktrace:
   [1] error(::String) at ./error.jl:33
   [2] assertscalar at /u/home/m/mleprovo/.julia/packages/GPUArrays/t8tJB/src/indexing.jl:8 [inlined]
   [3] setindex!(::CuArray{Int64,1}, ::Int64, ::Int64) at /u/home/m/mleprovo/.julia/packages/GPUArrays/t8tJB/src/indexing.jl:55
   [4] #mapslices#105(::Int64, ::Function, ::getfield(Flux, Symbol("##31#32")){Tuple{}}, ::CuArray{Float32,2}) at ./abstractarray.jl:1925
   [5] #mapslices at ./none:0 [inlined]
   [6] onecold(::CuArray{Float32,2}) at /u/home/m/mleprovo/.julia/packages/Flux/8XpDt/src/onehot.jl:78
   [7] macro expansion at /u/home/m/mleprovo/.julia/packages/Flux/8XpDt/test/cuda/cuda.jl:44 [inlined]
   [8] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [9] top-level scope at /u/home/m/mleprovo/.julia/packages/Flux/8XpDt/test/cuda/cuda.jl:42
   [10] include at ./boot.jl:317 [inlined]
   [11] include_relative(::Module, ::String) at ./loading.jl:1041
   [12] include(::Module, ::String) at ./sysimg.jl:29
   [13] include(::String) at ./client.jl:388
   [14] macro expansion at ./logging.jl:321 [inlined]
   [15] macro expansion at /u/home/m/mleprovo/.julia/packages/Flux/8XpDt/test/runtests.jl:11 [inlined]
   [16] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [17] top-level scope at /u/home/m/mleprovo/.julia/packages/Flux/8XpDt/test/runtests.jl:11
   [18] include at ./boot.jl:317 [inlined]
   [19] include_relative(::Module, ::String) at ./loading.jl:1041
   [20] include(::Module, ::String) at ./sysimg.jl:29
   [21] include(::String) at ./client.jl:388
   [22] top-level scope at none:0
   [23] eval(::Module, ::Any) at ./boot.jl:319
   [24] macro expansion at ./logging.jl:317 [inlined]
   [25] exec_options(::Base.JLOptions) at ./client.jl:219
   [26] _start() at ./client.jl:421

[ Info: Testing Flux/CUDNN

Test Summary:     | Pass  Error  Total
Flux              |  465      1    466
  Throttle        |   11            11
  Jacobian        |    1             1
  Initialization  |   12            12
  Params          |    2             2
  Precision       |    6             6
  Stacking        |    3             3
  onecold         |    4             4
  Optimise        |   11            11
  Optimiser       |    3             3
  Training Loop   |    2             2
  basic           |   17            17
  Dropout         |    8             8
  BatchNorm       |   13            13
  losses          |   30            30
  Pooling         |    2             2
  CNN             |    1             1
  Depthwise Conv  |    2             2
  Tracker         |  261           261
  CuArrays        |    8             8
  onecold gpu     |           1      1
  CUDNN BatchNorm |   10            10
  RNN             |   50            50

ERROR: LoadError: Some tests did not pass: 465 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /u/home/m/mleprovo/.julia/packages/Flux/8XpDt/test/runtests.jl:9

Package Flux errored during testing

Stacktrace:
 [1] pkgerror(::String, ::Vararg{String,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Types.jl:120
 [2] macro expansion at ./logging.jl:313 [inlined]
 [3] #test#61(::Bool, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1284
 [4] #test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:0 [inlined]
 [5] #test#42(::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:273
 [6] test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:258 [inlined]
 [7] #test#41 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:255 [inlined]
 [8] test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:255 [inlined]
 [9] #test#40 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:254 [inlined]
 [10] test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:254 [inlined]
 [11] #test#39 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:253 [inlined]
 [12] test(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:253
 [13] top-level scope at In[2]:2
MikeInnes commented 5 years ago

Can you test on master? This may just be a bad test that we removed.

mleprovost commented 5 years ago

Hello, I test on the Master branch and I still have errors in the testing. Is there a stable commit I can use?

[mleprovo@login2 Flux]$  git status
# On branch master
nothing to commit (working directory clean)
Pkg.test("Flux")
 Testing Flux
 Resolving package versions...
    Status `/tmp/tmp9XEd5L/Manifest.toml`
  [1520ce14] AbstractTrees v0.2.1
  [79e6a3ab] Adapt v0.4.2
  [9e28174c] BinDeps v0.8.10
  [b99e7846] BinaryProvider v0.5.3
  [944b1d66] CodecZlib v0.5.1
  [3da002f7] ColorTypes v0.7.5
  [5ae59095] Colors v0.9.5
  [bbf7d656] CommonSubexpressions v0.2.0
  [34da2185] Compat v1.5.1
  [864edb3b] DataStructures v0.15.0
  [163ba53b] DiffResults v0.0.4
  [b552c78f] DiffRules v0.0.10
  [53c48c17] FixedPointNumbers v0.5.3
  [587475ba] Flux v0.7.3+ [`~/.julia/environments/v1.0/dev/Flux`]
  [f6369f11] ForwardDiff v0.10.3
  [e5e0dc1b] Juno v0.5.4
  [1914dd2f] MacroTools v0.4.5
  [e89f7d12] Media v0.5.0
  [e1d29d7a] Missings v0.4.0
  [872c559c] NNlib v0.4.3
  [77ba4419] NaNMath v0.3.2
  [bac558e1] OrderedCollections v1.0.2
  [189a3867] Reexport v0.2.0
  [ae029012] Requires v0.5.2
  [a2af1166] SortingAlgorithms v0.3.1
  [276daf66] SpecialFunctions v0.7.2
  [90137ffa] StaticArrays v0.10.2
  [2913bbd2] StatsBase v0.28.1
  [3bb67fe8] TranscodingStreams v0.8.1
  [30578b45] URIParser v0.4.0
  [a5390f91] ZipFile v0.8.0
  [2a0f44e3] Base64  [`@stdlib/Base64`]
  [ade2ca70] Dates  [`@stdlib/Dates`]
  [8bb1440f] DelimitedFiles  [`@stdlib/DelimitedFiles`]
  [8ba89e20] Distributed  [`@stdlib/Distributed`]
  [b77e0a4c] InteractiveUtils  [`@stdlib/InteractiveUtils`]
  [76f85450] LibGit2  [`@stdlib/LibGit2`]
  [8f399da3] Libdl  [`@stdlib/Libdl`]
  [37e2e46d] LinearAlgebra  [`@stdlib/LinearAlgebra`]
  [56ddb016] Logging  [`@stdlib/Logging`]
  [d6f4376e] Markdown  [`@stdlib/Markdown`]
  [a63ad114] Mmap  [`@stdlib/Mmap`]
  [44cfe95a] Pkg  [`@stdlib/Pkg`]
  [de0858da] Printf  [`@stdlib/Printf`]
  [9abbd945] Profile  [`@stdlib/Profile`]
  [3fa0cd96] REPL  [`@stdlib/REPL`]
  [9a3f8284] Random  [`@stdlib/Random`]
  [ea8e919c] SHA  [`@stdlib/SHA`]
  [9e88b42a] Serialization  [`@stdlib/Serialization`]
  [1a1011a3] SharedArrays  [`@stdlib/SharedArrays`]
  [6462fe0b] Sockets  [`@stdlib/Sockets`]
  [2f01184e] SparseArrays  [`@stdlib/SparseArrays`]
  [10745b16] Statistics  [`@stdlib/Statistics`]
  [8dfed614] Test  [`@stdlib/Test`]
  [cf7118a7] UUIDs  [`@stdlib/UUIDs`]
  [4ec0a83e] Unicode  [`@stdlib/Unicode`]

[ Info: Testing Basics
[ Info: Downloading CMUDict dataset
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 3629k  100 3629k    0     0  1693k      0  0:00:02  0:00:02 --:--:-- 4184k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
127   382  127   382    0     0    427      0 --:--:-- --:--:-- --:--:--   427
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
140   281  140   281    0     0    304      0 --:--:-- --:--:-- --:--:--   304
[ Info: Downloading MNIST dataset
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 9680k  100 9680k    0     0  3215k      0  0:00:03  0:00:03 --:--:-- 5513k
[ Info: Downloading MNIST dataset
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 28881  100 28881    0     0  34914      0 --:--:-- --:--:-- --:--:--  414k
[ Info: Downloading MNIST dataset
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1610k  100 1610k    0     0  1003k      0  0:00:01  0:00:01 --:--:-- 2298k
[ Info: Downloading MNIST dataset
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4542  100  4542    0     0   5857      0 --:--:-- --:--:-- --:--:--  5857
[ Info: Downloading Fashion-MNIST dataset
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 25.1M  100 25.1M    0     0  4966k      0  0:00:05  0:00:05 --:--:-- 5437k
[ Info: Downloading Fashion-MNIST dataset
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 29515  100 29515    0     0  58156      0 --:--:-- --:--:-- --:--:-- 84328
[ Info: Downloading Fashion-MNIST dataset
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4318k  100 4318k    0     0  1759k      0  0:00:02  0:00:02 --:--:-- 1879k
[ Info: Downloading Fashion-MNIST dataset
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5148  100  5148    0     0  14348      0 --:--:-- --:--:-- --:--:-- 26131
[ Info: Downloading sentiment treebank dataset
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  771k  100  771k    0     0   455k      0  0:00:01  0:00:01 --:--:-- 1970k
[ Info: Testing Layers
[ Info: Running Gradient Checks

Tracker: Error During Test at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:196
  Test threw exception
  Expression: gradtest(conv, rand(10, 3, 2), randn(Float64, 2, 3, 2))
  MethodError: no method matching ∇conv_data(::Array{Float64,3}, ::Array{Float64,3}; size=(10, 3, 2))
  Closest candidates are:
    ∇conv_data(::A<:AbstractArray, ::A<:AbstractArray, !Matched::A<:AbstractArray; pad, stride, dilation, flipkernel) where A<:AbstractArray at /u/home/m/mleprovo/.julia/packages/NNlib/x0XUf/src/conv.jl:39 got unsupported keyword argument "size"
    ∇conv_data(::AbstractArray, !Matched::TrackedArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:407
    ∇conv_data(!Matched::TrackedArray, ::AbstractArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:408
  Stacktrace:
   [1] (::getfield(Flux.Tracker, Symbol("##503#504")){Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},TrackedArray{…,Array{Float64,3}},TrackedArray{…,Array{Float64,3}}})(::Array{Float64,3}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:402
   [2] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##503#504")){Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},TrackedArray{…,Array{Float64,3}},TrackedArray{…,Array{Float64,3}}},Tuple{Flux.Tracker.Tracked{Array{Float64,3}},Flux.Tracker.Tracked{Array{Float64,3}}}}, ::Array{Float64,3}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:23
   [3] back(::Flux.Tracker.Tracked{Array{Float64,3}}, ::Array{Float64,3}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [4] (::getfield(Flux.Tracker, Symbol("##3#4")){Bool})(::Flux.Tracker.Tracked{Array{Float64,3}}, ::Array{Float64,3}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [5] foreach at ./abstractarray.jl:1836 [inlined]
   [6] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("#back#526")){1,typeof(sin),Tuple{TrackedArray{…,Array{Float64,3}}}},Tuple{Flux.Tracker.Tracked{Array{Float64,3}}}}, ::Array{Float64,3}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [7] back(::Flux.Tracker.Tracked{Array{Float64,3}}, ::Array{Float64,3}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [8] (::getfield(Flux.Tracker, Symbol("##3#4")){Bool})(::Flux.Tracker.Tracked{Array{Float64,3}}, ::Array{Float64,3}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [9] foreach at ./abstractarray.jl:1836 [inlined]
   [10] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##460#461")){TrackedArray{…,Array{Float64,3}}},Tuple{Flux.Tracker.Tracked{Array{Float64,3}}}}, ::Float64, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [11] back(::Flux.Tracker.Tracked{Float64}, ::Int64, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [12] #back!#5 at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:65 [inlined]
   [13] #back! at ./none:0 [inlined]
   [14] #back!#27 at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/real.jl:16 [inlined]
   [15] back!(::TrackedReal{Float64}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/real.jl:14
   [16] gradient_(::Function, ::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:73
   [17] #gradient#24(::Bool, ::Function, ::Function, ::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:182
   [18] gradient(::Function, ::Array{Float64,3}, ::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:182
   [19] gradcheck(::Function, ::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:16
   [20] gradtest(::Function, ::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [21] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1168 [inlined]
   [22] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1093 [inlined]
   [23] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:42 [inlined]
   [24] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [25] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:14
Tracker: Error During Test at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:197
  Test threw exception
  Expression: gradtest(conv, rand(10, 10, 3, 2), randn(Float64, 2, 2, 3, 2))
  MethodError: no method matching ∇conv_data(::Array{Float64,4}, ::Array{Float64,4}; size=(10, 10, 3, 2))
  Closest candidates are:
    ∇conv_data(::A<:AbstractArray, ::A<:AbstractArray, !Matched::A<:AbstractArray; pad, stride, dilation, flipkernel) where A<:AbstractArray at /u/home/m/mleprovo/.julia/packages/NNlib/x0XUf/src/conv.jl:39 got unsupported keyword argument "size"
    ∇conv_data(::AbstractArray, !Matched::TrackedArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:407
    ∇conv_data(!Matched::TrackedArray, ::AbstractArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:408
  Stacktrace:
   [1] (::getfield(Flux.Tracker, Symbol("##503#504")){Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},TrackedArray{…,Array{Float64,4}},TrackedArray{…,Array{Float64,4}}})(::Array{Float64,4}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:402
   [2] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##503#504")){Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},TrackedArray{…,Array{Float64,4}},TrackedArray{…,Array{Float64,4}}},Tuple{Flux.Tracker.Tracked{Array{Float64,4}},Flux.Tracker.Tracked{Array{Float64,4}}}}, ::Array{Float64,4}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:23
   [3] back(::Flux.Tracker.Tracked{Array{Float64,4}}, ::Array{Float64,4}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [4] (::getfield(Flux.Tracker, Symbol("##3#4")){Bool})(::Flux.Tracker.Tracked{Array{Float64,4}}, ::Array{Float64,4}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [5] foreach at ./abstractarray.jl:1836 [inlined]
   [6] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("#back#526")){1,typeof(sin),Tuple{TrackedArray{…,Array{Float64,4}}}},Tuple{Flux.Tracker.Tracked{Array{Float64,4}}}}, ::Array{Float64,4}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [7] back(::Flux.Tracker.Tracked{Array{Float64,4}}, ::Array{Float64,4}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [8] (::getfield(Flux.Tracker, Symbol("##3#4")){Bool})(::Flux.Tracker.Tracked{Array{Float64,4}}, ::Array{Float64,4}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [9] foreach at ./abstractarray.jl:1836 [inlined]
   [10] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##460#461")){TrackedArray{…,Array{Float64,4}}},Tuple{Flux.Tracker.Tracked{Array{Float64,4}}}}, ::Float64, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [11] back(::Flux.Tracker.Tracked{Float64}, ::Int64, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [12] #back!#5 at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:65 [inlined]
   [13] #back! at ./none:0 [inlined]
   [14] #back!#27 at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/real.jl:16 [inlined]
   [15] back!(::TrackedReal{Float64}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/real.jl:14
   [16] gradient_(::Function, ::Array{Float64,4}, ::Vararg{Array{Float64,4},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:73
   [17] #gradient#24 at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:182 [inlined]
   [18] gradient(::Function, ::Array{Float64,4}, ::Array{Float64,4}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:182
   [19] gradcheck(::Function, ::Array{Float64,4}, ::Vararg{Array{Float64,4},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:16
   [20] gradtest(::Function, ::Array{Float64,4}, ::Vararg{Array{Float64,4},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [21] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1169 [inlined]
   [22] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1093 [inlined]
   [23] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:42 [inlined]
   [24] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [25] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:14
Tracker: Error During Test at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:198
  Test threw exception
  Expression: gradtest(conv, rand(10, 10, 10, 3, 2), randn(Float64, 2, 2, 2, 3, 2))
  MethodError: no method matching ∇conv_data(::Array{Float64,5}, ::Array{Float64,5}; size=(10, 10, 10, 3, 2))
  Closest candidates are:
    ∇conv_data(::A<:AbstractArray, ::A<:AbstractArray, !Matched::A<:AbstractArray; pad, stride, dilation, flipkernel) where A<:AbstractArray at /u/home/m/mleprovo/.julia/packages/NNlib/x0XUf/src/conv.jl:39 got unsupported keyword argument "size"
    ∇conv_data(::AbstractArray, !Matched::TrackedArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:407
    ∇conv_data(!Matched::TrackedArray, ::AbstractArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:408
  Stacktrace:
   [1] (::getfield(Flux.Tracker, Symbol("##503#504")){Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},TrackedArray{…,Array{Float64,5}},TrackedArray{…,Array{Float64,5}}})(::Array{Float64,5}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:402
   [2] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##503#504")){Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},TrackedArray{…,Array{Float64,5}},TrackedArray{…,Array{Float64,5}}},Tuple{Flux.Tracker.Tracked{Array{Float64,5}},Flux.Tracker.Tracked{Array{Float64,5}}}}, ::Array{Float64,5}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:23
   [3] back(::Flux.Tracker.Tracked{Array{Float64,5}}, ::Array{Float64,5}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [4] (::getfield(Flux.Tracker, Symbol("##3#4")){Bool})(::Flux.Tracker.Tracked{Array{Float64,5}}, ::Array{Float64,5}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [5] foreach at ./abstractarray.jl:1836 [inlined]
   [6] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("#back#526")){1,typeof(sin),Tuple{TrackedArray{…,Array{Float64,5}}}},Tuple{Flux.Tracker.Tracked{Array{Float64,5}}}}, ::Array{Float64,5}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [7] back(::Flux.Tracker.Tracked{Array{Float64,5}}, ::Array{Float64,5}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [8] (::getfield(Flux.Tracker, Symbol("##3#4")){Bool})(::Flux.Tracker.Tracked{Array{Float64,5}}, ::Array{Float64,5}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [9] foreach at ./abstractarray.jl:1836 [inlined]
   [10] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##460#461")){TrackedArray{…,Array{Float64,5}}},Tuple{Flux.Tracker.Tracked{Array{Float64,5}}}}, ::Float64, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [11] back(::Flux.Tracker.Tracked{Float64}, ::Int64, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [12] #back!#5 at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:65 [inlined]
   [13] #back! at ./none:0 [inlined]
   [14] #back!#27 at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/real.jl:16 [inlined]
   [15] back!(::TrackedReal{Float64}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/real.jl:14
   [16] gradient_(::Function, ::Array{Float64,5}, ::Vararg{Array{Float64,5},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:73
   [17] #gradient#24 at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:182 [inlined]
   [18] gradient(::Function, ::Array{Float64,5}, ::Array{Float64,5}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:182
   [19] gradcheck(::Function, ::Array{Float64,5}, ::Vararg{Array{Float64,5},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:16
   [20] gradtest(::Function, ::Array{Float64,5}, ::Vararg{Array{Float64,5},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [21] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1170 [inlined]
   [22] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1093 [inlined]
   [23] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:42 [inlined]
   [24] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [25] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:14
Tracker: Error During Test at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:200
  Test threw exception
  Expression: gradtest(∇conv_data, rand(10, 3, 2), randn(Float64, 2, 2, 3))
  MethodError: no method matching ∇conv_data(::Array{Float64,3}, ::Array{Float64,3})
  Closest candidates are:
    ∇conv_data(::A<:AbstractArray, ::A<:AbstractArray, !Matched::A<:AbstractArray; pad, stride, dilation, flipkernel) where A<:AbstractArray at /u/home/m/mleprovo/.julia/packages/NNlib/x0XUf/src/conv.jl:39
    ∇conv_data(::AbstractArray, !Matched::TrackedArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:407
    ∇conv_data(!Matched::TrackedArray, ::AbstractArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:408
  Stacktrace:
   [1] (::getfield(Main, Symbol("##39#40")){typeof(∇conv_data)})(::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [2] ngradient(::Function, ::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:7
   [3] gradcheck(::Function, ::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:16
   [4] gradtest(::Function, ::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [5] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1171 [inlined]
   [6] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1093 [inlined]
   [7] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:42 [inlined]
   [8] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [9] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:14
Tracker: Error During Test at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:201
  Test threw exception
  Expression: gradtest(∇conv_data, rand(10, 10, 3, 2), randn(Float64, 2, 2, 2, 3))
  MethodError: no method matching ∇conv_data(::Array{Float64,4}, ::Array{Float64,4})
  Closest candidates are:
    ∇conv_data(::A<:AbstractArray, ::A<:AbstractArray, !Matched::A<:AbstractArray; pad, stride, dilation, flipkernel) where A<:AbstractArray at /u/home/m/mleprovo/.julia/packages/NNlib/x0XUf/src/conv.jl:39
    ∇conv_data(::AbstractArray, !Matched::TrackedArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:407
    ∇conv_data(!Matched::TrackedArray, ::AbstractArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:408
  Stacktrace:
   [1] (::getfield(Main, Symbol("##39#40")){typeof(∇conv_data)})(::Array{Float64,4}, ::Vararg{Array{Float64,4},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [2] ngradient(::Function, ::Array{Float64,4}, ::Vararg{Array{Float64,4},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:7
   [3] gradcheck(::Function, ::Array{Float64,4}, ::Vararg{Array{Float64,4},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:16
   [4] gradtest(::Function, ::Array{Float64,4}, ::Vararg{Array{Float64,4},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [5] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1172 [inlined]
   [6] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1093 [inlined]
   [7] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:42 [inlined]
   [8] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [9] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:14
Tracker: Error During Test at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:202
  Test threw exception
  Expression: gradtest(∇conv_data, rand(10, 10, 10, 3, 2), randn(Float64, 2, 2, 2, 2, 3))
  MethodError: no method matching ∇conv_data(::Array{Float64,5}, ::Array{Float64,5})
  Closest candidates are:
    ∇conv_data(::A<:AbstractArray, ::A<:AbstractArray, !Matched::A<:AbstractArray; pad, stride, dilation, flipkernel) where A<:AbstractArray at /u/home/m/mleprovo/.julia/packages/NNlib/x0XUf/src/conv.jl:39
    ∇conv_data(::AbstractArray, !Matched::TrackedArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:407
    ∇conv_data(!Matched::TrackedArray, ::AbstractArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:408
  Stacktrace:
   [1] (::getfield(Main, Symbol("##39#40")){typeof(∇conv_data)})(::Array{Float64,5}, ::Vararg{Array{Float64,5},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [2] ngradient(::Function, ::Array{Float64,5}, ::Vararg{Array{Float64,5},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:7
   [3] gradcheck(::Function, ::Array{Float64,5}, ::Vararg{Array{Float64,5},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:16
   [4] gradtest(::Function, ::Array{Float64,5}, ::Vararg{Array{Float64,5},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [5] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1142 [inlined]
   [6] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1093 [inlined]
   [7] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:42 [inlined]
   [8] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [9] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:14
Tracker: Error During Test at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:206
  Test threw exception
  Expression: gradtest(∇conv_data, rand(10, 3, 2), randn(Float64, 2, 2, 3))
  MethodError: no method matching ∇conv_data(::Array{Float64,3}, ::Array{Float64,3})
  Closest candidates are:
    ∇conv_data(::A<:AbstractArray, ::A<:AbstractArray, !Matched::A<:AbstractArray; pad, stride, dilation, flipkernel) where A<:AbstractArray at /u/home/m/mleprovo/.julia/packages/NNlib/x0XUf/src/conv.jl:39
    ∇conv_data(::AbstractArray, !Matched::TrackedArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:407
    ∇conv_data(!Matched::TrackedArray, ::AbstractArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:408
  Stacktrace:
   [1] (::getfield(Main, Symbol("##39#40")){typeof(∇conv_data)})(::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [2] ngradient(::Function, ::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:7
   [3] gradcheck(::Function, ::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:16
   [4] gradtest(::Function, ::Array{Float64,3}, ::Vararg{Array{Float64,3},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [5] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1146 [inlined]
   [6] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1093 [inlined]
   [7] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:42 [inlined]
   [8] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [9] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:14
Tracker: Error During Test at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:207
  Test threw exception
  Expression: gradtest(∇conv_data, rand(10, 10, 3, 2), randn(Float64, 2, 2, 2, 3))
  MethodError: no method matching ∇conv_data(::Array{Float64,4}, ::Array{Float64,4})
  Closest candidates are:
    ∇conv_data(::A<:AbstractArray, ::A<:AbstractArray, !Matched::A<:AbstractArray; pad, stride, dilation, flipkernel) where A<:AbstractArray at /u/home/m/mleprovo/.julia/packages/NNlib/x0XUf/src/conv.jl:39
    ∇conv_data(::AbstractArray, !Matched::TrackedArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:407
    ∇conv_data(!Matched::TrackedArray, ::AbstractArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:408
  Stacktrace:
   [1] (::getfield(Main, Symbol("##39#40")){typeof(∇conv_data)})(::Array{Float64,4}, ::Vararg{Array{Float64,4},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [2] ngradient(::Function, ::Array{Float64,4}, ::Vararg{Array{Float64,4},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:7
   [3] gradcheck(::Function, ::Array{Float64,4}, ::Vararg{Array{Float64,4},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:16
   [4] gradtest(::Function, ::Array{Float64,4}, ::Vararg{Array{Float64,4},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [5] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1147 [inlined]
   [6] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1093 [inlined]
   [7] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:42 [inlined]
   [8] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [9] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:14
Tracker: Error During Test at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:208
  Test threw exception
  Expression: gradtest(∇conv_data, rand(10, 10, 10, 3, 2), randn(Float64, 2, 2, 2, 2, 3))
  MethodError: no method matching ∇conv_data(::Array{Float64,5}, ::Array{Float64,5})
  Closest candidates are:
    ∇conv_data(::A<:AbstractArray, ::A<:AbstractArray, !Matched::A<:AbstractArray; pad, stride, dilation, flipkernel) where A<:AbstractArray at /u/home/m/mleprovo/.julia/packages/NNlib/x0XUf/src/conv.jl:39
    ∇conv_data(::AbstractArray, !Matched::TrackedArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:407
    ∇conv_data(!Matched::TrackedArray, ::AbstractArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:408
  Stacktrace:
   [1] (::getfield(Main, Symbol("##39#40")){typeof(∇conv_data)})(::Array{Float64,5}, ::Vararg{Array{Float64,5},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [2] ngradient(::Function, ::Array{Float64,5}, ::Vararg{Array{Float64,5},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:7
   [3] gradcheck(::Function, ::Array{Float64,5}, ::Vararg{Array{Float64,5},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/numeric.jl:16
   [4] gradtest(::Function, ::Array{Float64,5}, ::Vararg{Array{Float64,5},N} where N) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:11
   [5] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1149 [inlined]
   [6] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1093 [inlined]
   [7] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:42 [inlined]
   [8] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [9] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/tracker.jl:14

[ Info: Testing GPU Support

CuArrays: Error During Test at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/cuda.jl:6
  Got exception outside of a @test
  MethodError: no method matching ∇conv_data(::CuArray{Float32,4}, ::CuArray{Float32,4}; size=(10, 10, 3, 2), stride=(1, 1), pad=(0, 0), dilation=(1, 1))
  Closest candidates are:
    ∇conv_data(::A<:AbstractArray, ::A<:AbstractArray, !Matched::A<:AbstractArray; pad, stride, dilation, flipkernel) where A<:AbstractArray at /u/home/m/mleprovo/.julia/packages/NNlib/x0XUf/src/conv.jl:39 got unsupported keyword argument "size"
    ∇conv_data(::AbstractArray, !Matched::TrackedArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:407
    ∇conv_data(!Matched::TrackedArray, ::AbstractArray; kw...) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:408
  Stacktrace:
   [1] (::getfield(Flux.Tracker, Symbol("##503#504")){Base.Iterators.Pairs{Symbol,Tuple{Int64,Int64},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}},CuArray{Float32,4},TrackedArray{…,CuArray{Float32,4}}})(::CuArray{Float32,4}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/array.jl:402
   [2] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##503#504")){Base.Iterators.Pairs{Symbol,Tuple{Int64,Int64},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}},CuArray{Float32,4},TrackedArray{…,CuArray{Float32,4}}},Tuple{Nothing,Flux.Tracker.Tracked{CuArray{Float32,4}}}}, ::CuArray{Float32,4}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:23
   [3] back(::Flux.Tracker.Tracked{CuArray{Float32,4}}, ::CuArray{Float32,4}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [4] foreach at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26 [inlined]
   [5] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("#back#526")){2,getfield(Base.Broadcast, Symbol("##26#28")){getfield(Base.Broadcast, Symbol("##27#29")){typeof(+),getfield(Base.Broadcast, Symbol("##9#10")){getfield(Base.Broadcast, Symbol("##9#10")){getfield(Base.Broadcast, Symbol("##11#12"))}},getfield(Base.Broadcast, Symbol("##13#14")){getfield(Base.Broadcast, Symbol("##13#14")){getfield(Base.Broadcast, Symbol("##15#16"))}},getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##3#4"))}}},typeof(identity)},Tuple{TrackedArray{…,CuArray{Float32,4}},TrackedArray{…,CuArray{Float32,4}}}},Tuple{Flux.Tracker.Tracked{CuArray{Float32,4}},Flux.Tracker.Tracked{CuArray{Float32,4}}}}, ::CuArray{Float32,4}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [6] back(::Flux.Tracker.Tracked{CuArray{Float32,4}}, ::CuArray{Float32,4}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [7] (::getfield(Flux.Tracker, Symbol("##3#4")){Bool})(::Flux.Tracker.Tracked{CuArray{Float32,4}}, ::CuArray{Float32,4}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [8] foreach at ./abstractarray.jl:1836 [inlined]
   [9] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##460#461")){TrackedArray{…,CuArray{Float32,4}}},Tuple{Flux.Tracker.Tracked{CuArray{Float32,4}}}}, ::Float32, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [10] back(::Flux.Tracker.Tracked{Float32}, ::Int64, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [11] #back!#5 at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:65 [inlined]
   [12] #back! at ./none:0 [inlined]
   [13] #back!#27 at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/real.jl:16 [inlined]
   [14] back!(::TrackedReal{Float32}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/lib/real.jl:14
   [15] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/cuda.jl:37 [inlined]
   [16] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [17] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/cuda.jl:8
   [18] include at ./boot.jl:317 [inlined]
   [19] include_relative(::Module, ::String) at ./loading.jl:1041
   [20] include(::Module, ::String) at ./sysimg.jl:29
   [21] include(::String) at ./client.jl:388
   [22] macro expansion at ./logging.jl:321 [inlined]
   [23] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/runtests.jl:11 [inlined]
   [24] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [25] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/runtests.jl:11
   [26] include at ./boot.jl:317 [inlined]
   [27] include_relative(::Module, ::String) at ./loading.jl:1041
   [28] include(::Module, ::String) at ./sysimg.jl:29
   [29] include(::String) at ./client.jl:388
   [30] top-level scope at none:0
   [31] eval(::Module, ::Any) at ./boot.jl:319
   [32] macro expansion at ./logging.jl:317 [inlined]
   [33] exec_options(::Base.JLOptions) at ./client.jl:219
   [34] _start() at ./client.jl:421

[ Info: Testing Flux/CUDNN

batch_size = 1: Error During Test at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:7
  Got exception outside of a @test
  CUDNNError(code 3, CUDNN_STATUS_BAD_PARAM)
  Stacktrace:
   [1] macro expansion at /u/home/m/mleprovo/.julia/packages/CuArrays/PD3UJ/src/dnn/error.jl:19 [inlined]
   [2] cudnnRNNBackwardData(::Flux.CUDA.RNNDesc{Float32}, ::Int64, ::Array{CuArrays.CUDNN.TensorDesc,1}, ::CuArray{Float32,1}, ::Array{CuArrays.CUDNN.TensorDesc,1}, ::CuArray{Float32,1}, ::CuArrays.CUDNN.TensorDesc, ::CuArray{Float32,1}, ::Ptr{Nothing}, ::Ptr{Nothing}, ::CuArrays.CUDNN.FilterDesc, ::CuArray{Float32,1}, ::CuArrays.CUDNN.TensorDesc, ::CuArray{Float32,1}, ::Ptr{Nothing}, ::Ptr{Nothing}, ::Array{CuArrays.CUDNN.TensorDesc,1}, ::CuArray{Float32,1}, ::CuArrays.CUDNN.TensorDesc, ::CuArray{Float32,1}, ::Ptr{Nothing}, ::Ptr{Nothing}, ::CuArray{UInt8,1}, ::CuArray{UInt8,1}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/cuda/curnn.jl:170
   [3] backwardData(::Flux.CUDA.RNNDesc{Float32}, ::CuArray{Float32,1}, ::CuArray{Float32,1}, ::CuArray{Float32,1}, ::Nothing, ::CuArray{Float32,1}, ::Nothing, ::CuArray{UInt8,1}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/cuda/curnn.jl:187
   [4] backwardData(::Flux.CUDA.RNNDesc{Float32}, ::CuArray{Float32,1}, ::CuArray{Float32,1}, ::CuArray{Float32,1}, ::CuArray{Float32,1}, ::CuArray{UInt8,1}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/cuda/curnn.jl:195
   [5] (::getfield(Flux.CUDA, Symbol("##8#9")){Flux.GRUCell{TrackedArray{…,CuArray{Float32,2}},TrackedArray{…,CuArray{Float32,1}}},TrackedArray{…,CuArray{Float32,1}},TrackedArray{…,CuArray{Float32,1}},CuArray{UInt8,1},Tuple{CuArray{Float32,1},CuArray{Float32,1}}})(::Tuple{CuArray{Float32,1},CuArray{Float32,1}}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/cuda/curnn.jl:306
   [6] back_(::Flux.Tracker.Call{getfield(Flux.CUDA, Symbol("##8#9")){Flux.GRUCell{TrackedArray{…,CuArray{Float32,2}},TrackedArray{…,CuArray{Float32,1}}},TrackedArray{…,CuArray{Float32,1}},TrackedArray{…,CuArray{Float32,1}},CuArray{UInt8,1},Tuple{CuArray{Float32,1},CuArray{Float32,1}}},Tuple{Flux.Tracker.Tracked{CuArray{Float32,1}},Flux.Tracker.Tracked{CuArray{Float32,1}},Flux.Tracker.Tracked{CuArray{Float32,2}},Flux.Tracker.Tracked{CuArray{Float32,2}},Flux.Tracker.Tracked{CuArray{Float32,1}}}}, ::Tuple{CuArray{Float32,1},CuArray{Float32,1}}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:23
   [7] back(::Flux.Tracker.Tracked{Tuple{CuArray{Float32,1},CuArray{Float32,1}}}, ::Tuple{CuArray{Float32,1},Int64}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [8] (::getfield(Flux.Tracker, Symbol("##3#4")){Bool})(::Flux.Tracker.Tracked{Tuple{CuArray{Float32,1},CuArray{Float32,1}}}, ::Tuple{CuArray{Float32,1},Int64}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [9] foreach(::Function, ::Tuple{Flux.Tracker.Tracked{Tuple{CuArray{Float32,1},CuArray{Float32,1}}},Nothing}, ::Tuple{Tuple{CuArray{Float32,1},Int64},Nothing}) at ./abstractarray.jl:1836
   [10] back_(::Flux.Tracker.Call{getfield(Flux.Tracker, Symbol("##356#358")){Flux.Tracker.TrackedTuple{Tuple{CuArray{Float32,1},CuArray{Float32,1}}},Int64},Tuple{Flux.Tracker.Tracked{Tuple{CuArray{Float32,1},CuArray{Float32,1}}},Nothing}}, ::CuArray{Float32,1}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:26
   [11] back(::Flux.Tracker.Tracked{CuArray{Float32,1}}, ::CuArray{Float32,1}, ::Bool) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:46
   [12] back!(::TrackedArray{…,CuArray{Float32,1}}, ::CuArray{Float32,1}) at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/src/tracker/back.jl:65
   [13] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:23 [inlined]
   [14] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
   [15] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:7 [inlined]
   [16] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
   [17] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:4 [inlined]
   [18] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [19] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:4
   [20] include at ./boot.jl:317 [inlined]
   [21] include_relative(::Module, ::String) at ./loading.jl:1041
   [22] include(::Module, ::String) at ./sysimg.jl:29
   [23] include(::String) at ./client.jl:388
   [24] top-level scope at logging.jl:308
   [25] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/cuda.jl:42
   [26] include at ./boot.jl:317 [inlined]
   [27] include_relative(::Module, ::String) at ./loading.jl:1041
   [28] include(::Module, ::String) at ./sysimg.jl:29
   [29] include(::String) at ./client.jl:388
   [30] macro expansion at ./logging.jl:321 [inlined]
   [31] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/runtests.jl:11 [inlined]
   [32] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [33] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/runtests.jl:11
   [34] include at ./boot.jl:317 [inlined]
   [35] include_relative(::Module, ::String) at ./loading.jl:1041
   [36] include(::Module, ::String) at ./sysimg.jl:29
   [37] include(::String) at ./client.jl:388
   [38] top-level scope at none:0
   [39] eval(::Module, ::Any) at ./boot.jl:319
   [40] macro expansion at ./logging.jl:317 [inlined]
   [41] exec_options(::Base.JLOptions) at ./client.jl:219
   [42] _start() at ./client.jl:421
batch_size = 5: Test Failed at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:26
  Expression: ((rnn.cell).Wi).grad ≈ collect(((curnn.cell).Wi).grad)
   Evaluated: Float32[0.0264221 0.0311623 … 0.0401529 0.0481648; -0.00580488 -0.00684245 … -0.000633113 -0.00512767; … ; -0.488995 -0.397307 … -0.543058 -0.593239; -1.59764 -1.72794 … -1.24722 -2.0625] ≈ Float32[0.0163969 0.0200817 … 0.0357488 0.036319; -0.00383122 -0.00466101 … 0.000233916 -0.0027956; … ; -0.655613 -0.581466 … -0.616253 -0.790113; -1.32217 -1.42347 … -1.1262 -1.737]
Stacktrace:
 [1] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:26 [inlined]
 [2] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [3] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:7 [inlined]
 [4] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [5] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:4 [inlined]
 [6] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
 [7] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:4
batch_size = 5: Test Failed at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:27
  Expression: ((rnn.cell).Wh).grad ≈ collect(((curnn.cell).Wh).grad)
   Evaluated: Float32[-0.0155654 0.00326925 … -0.0217709 -0.00794538; 0.00331486 -0.000961465 … 0.00208863 -0.000682507; … ; 0.0600955 -0.0379895 … 0.0877803 0.056216; 0.13519 -0.049922 … 0.1269 0.0464798] ≈ Float32[-0.0102084 0.000635069 … -0.0177858 -0.00776742; 0.00226024 -0.000442875 … 0.00130408 -0.000717541; … ; 0.0800567 -0.047805 … 0.10263 0.0568791; 0.120198 -0.04255 … 0.115747 0.0459818]
Stacktrace:
 [1] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:27 [inlined]
 [2] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [3] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:7 [inlined]
 [4] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [5] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:4 [inlined]
 [6] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
 [7] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:4
batch_size = 5: Test Failed at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:28
  Expression: ((rnn.cell).b).grad ≈ collect(((curnn.cell).b).grad)
   Evaluated: Float32[0.0431294, -0.00524064, 0.00328069, 0.0383973, -0.0277038, -0.190451, 0.378964, 0.113537, -1.10465, -0.109885, -0.380812, -0.806893, -0.591657, -0.840546, -2.16732] ≈ Float32[0.0309576, -0.0028444, 0.00784941, 0.0508012, -0.0251556, -0.116478, 0.196908, 0.156512, -1.30342, -0.109, -0.0338722, -0.537076, -0.840815, -1.04284, -1.83286]
Stacktrace:
 [1] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:28 [inlined]
 [2] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [3] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:7 [inlined]
 [4] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [5] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:4 [inlined]
 [6] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
 [7] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:4
batch_size = 5: Test Failed at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:29
  Expression: ((rnn.cell).h).grad ≈ collect(((curnn.cell).h).grad)
   Evaluated: Float32[-0.0128217, -0.706767, -0.358611, -1.41596, -0.547122] ≈ Float32[0.0667142, -0.127551, -0.454483, -1.79769, -0.414044]
Stacktrace:
 [1] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:29 [inlined]
 [2] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [3] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:7 [inlined]
 [4] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [5] macro expansion at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:4 [inlined]
 [6] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
 [7] top-level scope at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/cuda/curnn.jl:4
Test Summary:            | Pass  Fail  Error  Total
Flux                     |  460     4     11    475
  Throttle               |   11                  11
  Jacobian               |    1                   1
  Initialization         |   12                  12
  Params                 |    2                   2
  Basic Stacking         |    1                   1
  Precision              |    6                   6
  Stacking               |    3                   3
  onecold                |    4                   4
  Optimise               |   11                  11
  Optimiser              |    3                   3
  Training Loop          |    2                   2
  basic                  |   17                  17
  Dropout                |    8                   8
  BatchNorm              |   14                  14
  losses                 |   30                  30
  Pooling                |    2                   2
  CNN                    |    1                   1
  Depthwise Conv         |    2                   2
  Tracker                |  264            9    273
    indexing & slicing   |                    No tests
    concat               |  154                 154
    mean                 |    5                   5
    maximum              |    5                   5
    minimum              |    5                   5
    equality & order     |   16                  16
    reshape              |    8                   8
    Intermediates        |    2                   2
    Fallbacks            |    1                   1
    collect              |    3                   3
    Hooks                |    1                   1
    Checkpointing        |    4                   4
    Updates              |    2                   2
    Params               |    1                   1
    Forward              |    2                   2
    Custom Sensitivities |    1                   1
  CuArrays               |    8            1      9
  CUDNN BatchNorm        |   10                  10
  RNN                    |   40     4      1     45
    R = Flux.RNN         |   16                  16
    R = Flux.GRU         |    6     4      1     11
      batch_size = 1     |    2            1      3
      batch_size = 5     |    4     4             8
    R = Flux.LSTM        |   18                  18

ERROR: LoadError: Some tests did not pass: 460 passed, 4 failed, 11 errored, 0 broken.
in expression starting at /u/home/m/mleprovo/.julia/environments/v1.0/dev/Flux/test/runtests.jl:9

Package Flux errored during testing

Stacktrace:
 [1] pkgerror(::String, ::Vararg{String,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Types.jl:120
 [2] macro expansion at ./logging.jl:313 [inlined]
 [3] #test#61(::Bool, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1284
 [4] #test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:0 [inlined]
 [5] #test#42(::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:273
 [6] test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:258 [inlined]
 [7] #test#41 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:255 [inlined]
 [8] test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:255 [inlined]
 [9] #test#40 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:254 [inlined]
 [10] test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:254 [inlined]
 [11] #test#39 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:253 [inlined]
 [12] test(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:253
 [13] top-level scope at In[5]:2
MikeInnes commented 5 years ago

Ah, you might need to figure out the right version of NNlib on master.

To be totally clear, the latest tag is fine and completely stable, there's just a spurious test failure that you needn't worry about.

mleprovost commented 5 years ago

Sorry, I am pretty novice to GPU computing how do I do to figure the right version of NNlib?

mleprovost commented 5 years ago

It seems that the last commit: Merge pull request #639 from ropenta/master introduces problems

I did a git reset to the commit just before: Added an example of Conv to Flux.jl/src/layers/conv.jl, and clarified…, and now Flux with GPU is working.

MikeInnes commented 5 years ago

Closing for now as it looks like you've got it figured out, and a new release will sort the underlying issue.