vcat of Knet.KnetArray{Float64} arrays produces Array{Float64}

[montyvesselinov]

vcat of Knet.KnetArray{Float64} arrays produces Array{Float64}; is this expected?!

[denizyuret]

Definitely not, which version of Knet?

On Sun, Mar 31, 2019 at 1:01 AM Velimir V Vesselinov (monty) < notifications@github.com> wrote:

vcat of Knet.KnetArray{Float64} arrays produces Array{Float64}; is this expected?!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/denizyuret/Knet.jl/issues/450, or mute the thread https://github.com/notifications/unsubscribe-auth/ABvNpvvDsyAA3m1YkXWmSeaTLEd5e2LLks5vb96egaJpZM4cT_ZP .

[montyvesselinov]

I use Knet v1.2.1

Actually, it is a different problem:

typeof(z) = Knet.KnetArray{Float64,2}
typeof(w[13]) = Knet.KnetArray{Float64,2}
typeof(y) = Knet.KnetArray{Float64,2}
x0 = vcat(z, w[13] * y)

produces Array of KnetArray instead of just KnetArray:

typeof(x0) = Array{Knet.KnetArray{Float64,2},1}

I do not think is right.

[montyvesselinov]

In fact, the vcat is not executed. just two Arrays are returned.

[montyvesselinov]

This is very weird.

if i try to replicate this in the REPL everything works:

a=Knet.KnetArray{Float64,2}(undef, 200, 100)
b=Knet.KnetArray{Float64,2}(undef, 36, 36)
c=Knet.KnetArray{Float64,2}(undef, 36, 100)
vcat(a, b * c)

however, inside the function call it does not:

function bsgan_gnet(w, z, y, m; training=true)
    x = w[13] * y
    @show typeof(x)
    @show typeof(z)
    x0 = Knet.vcat(z, x)
    @show typeof(x0)
end

typeof(z) = Knet.KnetArray{Float64,2}
typeof(x) = Knet.KnetArray{Float64,2}
typeof(x0) = Array{Knet.KnetArray{Float64,2},1}

[denizyuret]

Unfortunately I cannot replicate your bug. Please send me a minimal but complete piece of code to replicate instead of incomplete snippets, the output of versioninfo(), the output of Pkg.status(), and the output of include(Knet.dir("test/gpu.jl")) so I can try to figure out what went wrong on your setup.

Alternatively you can try to see if the bug is also replicated in the official AWS Image and in the latest Knet master (which you can try using pkg"dev AutoGrad Knet").

[kadir-gunel]

Hello, I am having the same issue with hcat. And I get this error :

no method matching hcat(::Array{KnetArray{Float32,2},2}, KnetArray{Float32,2})

I share with you my system information and the minimal code snippet below in order to reproduce the error.

versioninfo():

Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
OS: macOS (x86_64-apple-darwin14.5.0)
CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Pkg> status

[c52e3926] Atom v0.8.4
[fbb218c0] BSON v0.2.3
[159f3aea] Cairo v0.5.6
[ebbdde9d] DiffEqBayes v1.1.0
[01453d9d] DiffEqDiffTools v0.8.1
[aae7a2af] DiffEqFlux v0.4.0
[0c46a032] DifferentialEquations v6.3.0
[c5bfea45] Embeddings v0.3.1
[587475ba] Flux v0.8.2
[c27321d9] Glob v1.2.0
[e5e0dc1b] Juno v0.7.0
[1902f260] Knet v1.2.1+ #master (https://github.com/denizyuret/Knet.jl.git)
[8ef0a80b] Languages v0.4.2
[5fb14364] OhMyREPL v0.5.1
[91a5bcdd] Plots v0.24.0
[92933f4c] ProgressMeter v0.9.0
[a2db99b7] TextAnalysis v0.6.0
[e0df1984] TextParse v0.9.0
[fce5fe82] Turing v0.6.14
[e88e6eb3] Zygote v0.2.0

Pkg> test Knet

Testing Knet
Resolving package versions...
Status `/var/folders/tp/qfx5zl550713q7mx9n6swv800000gn/T/tmpZEsrcl/Manifest.toml`
[6710c13c] AutoGrad v1.1.3
[9e28174c] BinDeps v0.8.10
[b99e7846] BinaryProvider v0.5.3
[3895d2a7] CUDAapi v0.6.3
[944b1d66] CodecZlib v0.5.2
[34da2185] Compat v2.1.0
[a8cc5b0e] Crayons v4.0.0
[864edb3b] DataStructures v0.15.0
[5789e2e9] FileIO v1.0.6
[033835bb] JLD2 v0.1.2
[1902f260] Knet v1.2.1+ #master (https://github.com/denizyuret/Knet.jl.git)
[bac558e1] OrderedCollections v1.1.0
[276daf66] SpecialFunctions v0.7.2
[a759f4b9] TimerOutputs v0.5.0
[3bb67fe8] TranscodingStreams v0.9.3
[30578b45] URIParser v0.4.0
[2a0f44e3] Base64  [`@stdlib/Base64`]
[ade2ca70] Dates  [`@stdlib/Dates`]
[8bb1440f] DelimitedFiles  [`@stdlib/DelimitedFiles`]
[8ba89e20] Distributed  [`@stdlib/Distributed`]
[b77e0a4c] InteractiveUtils  [`@stdlib/InteractiveUtils`]
[76f85450] LibGit2  [`@stdlib/LibGit2`]
[8f399da3] Libdl  [`@stdlib/Libdl`]
[37e2e46d] LinearAlgebra  [`@stdlib/LinearAlgebra`]
[56ddb016] Logging  [`@stdlib/Logging`]
[d6f4376e] Markdown  [`@stdlib/Markdown`]
[a63ad114] Mmap  [`@stdlib/Mmap`]
[44cfe95a] Pkg  [`@stdlib/Pkg`]
[de0858da] Printf  [`@stdlib/Printf`]
[3fa0cd96] REPL  [`@stdlib/REPL`]
[9a3f8284] Random  [`@stdlib/Random`]
[ea8e919c] SHA  [`@stdlib/SHA`]
[9e88b42a] Serialization  [`@stdlib/Serialization`]
[1a1011a3] SharedArrays  [`@stdlib/SharedArrays`]
[6462fe0b] Sockets  [`@stdlib/Sockets`]
[2f01184e] SparseArrays  [`@stdlib/SparseArrays`]
[10745b16] Statistics  [`@stdlib/Statistics`]
[8dfed614] Test  [`@stdlib/Test`]
[cf7118a7] UUIDs  [`@stdlib/UUIDs`]
[4ec0a83e] Unicode  [`@stdlib/Unicode`]
Test Summary: | Pass  Total
kptr:alloc    |    8      8
Test Summary: | Pass  Total
kptr:gc       |    7      7
Test Summary: | Pass  Total
kptr:realloc  |    8      8
4.392940 seconds (13.07 M allocations: 745.202 MiB, 5.35% gc time)
Knet.gpuCount() = 2
Knet.gpu() = 0
Knet.tk = ["/Developer/NVIDIA/CUDA-9.0", "/Users/phd", "/usr/local/cuda"]
Knet.libknet8 = "/Users/phd/.julia/packages/Knet/CvLPF/deps/libknet8"
Knet.cudartfound = true
Knet.cudaRuntimeVersion = 9000
Knet.cudaDriverVersion = 9010
Knet.cudaGetDeviceCount() = 2
Knet.cudaGetDevice() = 0
Knet.cudaMemGetInfo() = (11598176256, 11810963456)
Knet.cudaDeviceSynchronize() = nothing
Knet.nvmlfound = false
Knet.cublashandle() = Ptr{Nothing} @0x00007f8b63b11000
Knet.cublasVersion = 9000
Knet.cudnnhandle() = Ptr{Nothing} @0x00007f8b5bfd5930
Knet.cudnnVersion = 7004
Knet.dir() = "/Users/phd/.julia/packages/Knet/CvLPF"
readdir(Knet.dir("deps")) = [".deprecated", ".gitignore", "Makefile", "README.windows", "build.jl", "build.log", "conv.cpp", "conv.o", "cuda01.cu", "cuda01.jl", "cuda01.o", "cuda1.cu", "cuda1.jl", "cuda1.o", "cuda11.cu", "cuda11.jl", "cuda11.o", "cuda12.cu", "cuda12.jl", "cuda12.o", "cuda13.cu", "cuda13.jl", "cuda13.o", "cuda14.jl", "cuda16.cu", "cuda16.jl", "cuda16.o", "cuda17.cu", "cuda17.jl", "cuda17.o", "cuda20.cu", "cuda20.jl", "cuda20.o", "cuda21.cu", "cuda21.jl", "cuda21.o", "cuda22.cu", "cuda22.jl", "cuda22.o", "foo.cpp", "gamma.jl", "libknet8.dylib"]
Test Summary: | Pass  Total
gpu           |   17     17
1.505799 seconds (1.89 M allocations: 92.500 MiB, 3.10% gc time)
Test Summary: | Pass  Total
distributions |    3      3
0.931003 seconds (4.42 M allocations: 219.500 MiB, 5.11% gc time)
Test Summary: | Pass  Total
dropout       |    3      3
1.420429 seconds (4.97 M allocations: 263.680 MiB, 3.62% gc time)
Test Summary: | Pass  Total
bmm           |   18     18
2.705537 seconds (9.60 M allocations: 486.736 MiB, 3.77% gc time)
Test Summary: | Pass  Total
JLD           |    4      4
12.088468 seconds (53.19 M allocations: 2.542 GiB, 8.18% gc time)
Test Summary: | Pass  Total
serialize     |    8      8
1.705445 seconds (7.08 M allocations: 362.306 MiB, 4.81% gc time)
Test Summary: | Pass  Total
loss          |  111    111
19.039356 seconds (66.50 M allocations: 3.354 GiB, 5.22% gc time)

Test Summary: | Pass Total linalg | 194 194 17.990060 seconds (54.99 M allocations: 2.768 GiB, 4.43% gc time) batchnorm | 65 65 14.500838 seconds (47.85 M allocations: 2.385 GiB, 5.63% gc time) Test Summary: | Pass Total karray | 316 316 19.321204 seconds (56.75 M allocations: 2.897 GiB, 5.77% gc time) Test Summary: | Pass Total conv | 139 139 31.375284 seconds (68.96 M allocations: 3.451 GiB, 29.54% gc time) Test Summary: | Pass Total reduction | 1620 1620 38.694604 seconds (128.74 M allocations: 5.919 GiB, 10.05% gc time) Test Summary: | Pass Total rnn | 2276 2276 47.626459 seconds (98.32 M allocations: 7.129 GiB, 7.49% gc time) Test Summary: | Pass Total update! | 20 20 ┌ Warning: optimizers is deprecated, use sgd, adam etc. instead. └ @ Knet ~/.julia/packages/Knet/CvLPF/src/update.jl:625 Test Summary: | Pass Total optimizers | 2 2 63.582404 seconds (88.71 M allocations: 3.607 GiB, 3.19% gc time) 14:08:05 binary: array-scalar 14:08:39 binary: literal-pow 14:08:39 binary: array-vector 14:09:05 binary: array-array 14:09:09 binary: ndims Test Summary: | Pass Total binary | 3780 3780 14:09:09 binary: done 64.243938 seconds (181.12 M allocations: 9.231 GiB, 8.47% gc time) Test Summary: | Pass Total unary | 1880 1880 94.469678 seconds (209.62 M allocations: 10.769 GiB, 8.15% gc time) Testing Knet tests passed

And the code sample  : 

```julia
using Knet

x = KnetArray(rand(Float32, 5,5))
y = KnetArray(rand(Float32, 5,5))

x = hcat(x*y, y)
julia> typeof(x)
KnetArray{Float32,2}

function my_foo(i,j)
    z = hcat(i, j)
    t = z .* z
    z = hcat(t, i)
    return z
end

x = KnetArray(rand(Float32, 5,5))
julia> typeof(x)
KnetArray{Float32,2}

x = my_foo(x,y)

# there are 2  KArrays inside the value x. 
julia> typeof(x)
Array{KnetArray{Float32,2},2}

# The first value in the array is the desired one
julia> x[1]
5×10 KnetArray{Float32,2}:
 0.00455322   0.380933     0.303049   …  0.00379012  0.0138041  0.385351  
 0.0558933    0.075582     0.844929      0.846572    0.16181    0.0670144 
 0.000286322  0.486351     0.0810923     0.932659    0.963254   0.00806545
 0.000392439  0.000632795  0.500208      0.684875    0.0825315  0.205481  
 0.535141     0.470819     0.862983      0.048351    0.272964   0.267019  

# The second one is the old value of x. 
julia> x[2]
5×5 KnetArray{Float32,2}:
 0.0674776  0.617198   0.550499  0.351458   0.268918 
 0.236418   0.274922   0.9192    0.865868   0.573511 
 0.016921   0.697389   0.284767  0.479399   0.0315462
 0.0198101  0.0251554  0.707254  0.450083   0.59201  
 0.731533   0.686163   0.928969  0.0971808  0.976873 

[kadir-gunel]

On the other hand, this behavior does not happen on my linux machine.

B.R.

[kirnap]

I have a macbook without a GPU and everything from that code snippet works perfectly fine

[kadir-gunel]

hi @kirnap but how can you create a KnetArray without gpu ?

[kirnap]

Yes you're right, I forgot to mention I've tested without a GPU and normal Array

[kadir-gunel]

Today I updated julia from 1.0.3 to 1.1.0 and I get the same error, which was happening on Mac, on a linux machine. The same code is working perfectly under 1.0.3 but not in 1.1.0.

Could someone please try to execute the code snippet above under 1.1.0 in order to validate ?