1.1.2 Todo List

[denizyuret]

• [ ] conv.jl memory management and algorithm selection issues. rnn algorithm selection.
  • [ ] check cuda 10 and cudnn 7.3 for performance improvements.
• [ ] test ekin's slowdown example (array overwrite).
• [ ] kptr check ages in mac model to see if free would be useful.
• [x] AutoGrad only uses broadcasted now, compare performance with using broadcast
• [ ] add using LinearAlgebra: lmul!, rmul! to test/linalg.jl
• [ ] Update docs and tests with new model interface and training related functions.
  • [ ] check test coverage.
  • [ ] Documenter.jl 0.20 breaking update.
• [ ] separate and move KnetArray to KnetML.
  • [ ] try making karray <: AbstractArray and overriding get/setindex.
  • [ ] test on other AD and GPUarray pkgs.
• [ ] eventually, slim down update! and rnn gpu tests
• [ ] reorganize unary.jl and broadcast.jl
  • [ ] unsafe_copy! is not in base any more? fix unsafe_copy!, unsafe_convert in karray looking in base
  • [ ] unary_nd, indexed_function, isequivalent, _dbg, ssize not in AutoGrad any more.
• [ ] check TODO.
• [ ] pull requests.
• [ ] reconsider Requires.jl.
• [ ] Knet/tutorial
  • [ ] 03.lin.ipynb: use this as a mini AutoGrad demo? start with functional style? (https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/knet-users/dwK1bPHV36o/nTLJoqpRCwAJ)
  • [ ] 05.cnn.ipynb: let user draw own digits.
  • [ ] 06.rnn.ipynb: add figure, manual rnn definition, keepstate reduces performance?
  • [ ] 08.charlm.ipynb: add the julia model.
  • [ ] 09.s2s.ipynb: convert to new interface, do real MT?
  • [ ] check prerequisites between notebooks
  • [ ] add optimization tutorial
• [ ] Knet/docs
  • [ ] backprop.md
  • [ ] cnn.md
  • [ ] gen.md
  • [ ] index.md
  • [ ] install.md
  • [ ] mlp.md
  • [ ] opt.md
  • [ ] reference.md
  • [ ] rl.md
  • [ ] rnn.md
  • [ ] softmax.md
  • [ ] tutorial.md
• [ ] Knet/examples
  • [ ] update all to use the new interface??
  • [ ] DeepLearningFrameworks: submit new results.
  • [ ] julia-tutorial: check old commands, turn into notebook.
• [ ] KnetML: check projects for Julia 1.0 compatibility
  • [ ] MAC-Network
  • [ ] NLPDemos
  • [ ] WGAN
  • [ ] Relational Network
  • [ ] Knet-the-Julia-dope
  • [ ] singularity-images
  • [ ] Neural-Style-Transfer
• [ ] Knet/prof
  • [ ] conv.jl: check regression in poolx(::Array), conv4x(::Array)
  • [ ] karray.jl: check regression in vcat(mat,row), vcat(row,mat), hcat(kcol,kmat), bad 0.6 tests for vcat(kmat,krow)
  • [ ] reduction.jl: check regression in sum(a,0), slight regression in sum(a,1) and sum(a,2)

[kirnap]

I think following features would make new RNN more safe:

• [ ] Knet/RNN
  • [ ] Check with declared input size is equal to used input size
  • [ ] Warn user to use Float32 for an input array X

[denizyuret]

Can you give examples?

[kirnap]

# suppose we are given
r = RNN(8, 10); 
# 1
x1 = KnetArray{Float32}(rand(8,1));
x2 = KnetArray{Float32}(rand(1128, 1));
r(x1);
r(x2); # why this call does not trigger any error?

# 2
# I know why it does not work  which is (r.dataType == Float32 by default),
r(KnetArray(rand(8,1)))  # throws not a user-friend error
# We may put somewhere in rnnforw @assert(r.dataType == eltype(input), "Conflicting types...")