search

XtractOpen / Meganet.jl

A fresh approach to deep learning written in Julia
http://www.xtract.ai/
Other
14 stars 9 forks source link

Add IP/OP function mapping to mean along dim, optimize batchnorm #48

Closed klensink closed 6 years ago

klensink commented 6 years ago

60% time, 50% memory reduction.

Can this function operate in place on Y? It would be nice not having to make a copy of Y.

coveralls commented 6 years ago

Coverage Status

Coverage decreased (-0.1%) to 78.804% when pulling f4cb05fd3ee4eea3136f42e056fa8d7ffb6e3063 on klensink:pull-request/d4e59c0e into a3c401af389a839236ba08feb8de4d15721dd908 on XtractOpen:dev.

eldadHaber commented 6 years ago

Not sure Try and see if we go through the derivative test

On Feb 7, 2018, at 5:53 PM, Keegan Lensink notifications@github.com wrote:

60% time, 50% memory reduction.

Can this function operate in place on Y? It would be nice not having to make a copy of Y.

You can view, comment on, or merge this pull request online at:

https://github.com/XtractOpen/Meganet.jl/pull/48

Commit Summary

Add IP/OP function mapping to mean along dim, optimize batchnorm File Changes

M benchmarks/micro/bm_batchnorm.jl (5) M benchmarks/micro/bm_batchnorm.jld (0) M src/layers/normLayer.jl (11) M src/utils/utilities.jl (27) Patch Links:

https://github.com/XtractOpen/Meganet.jl/pull/48.patch https://github.com/XtractOpen/Meganet.jl/pull/48.diff — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

klensink commented 6 years ago

Good point, I'll add some

On Feb 10, 2018 11:44 AM, "Lars Ruthotto" notifications@github.com wrote:

@lruthotto commented on this pull request.

Do we want to add unit testing for those mean functions?

In src/utils/utilities.jl https://github.com/XtractOpen/Meganet.jl/pull/48#discussion_r167409102:

+Apply the function f to each element of A, and compute the mean along dimension in region. +""" +function Base.mean(f::Function, a::AbstractArray, region::Int)

  • x = Base.mapreducedim(f, +, a, region)
  • n = max(1, Base._length(x)) // Base._length(a)
  • x .= x .* n
  • return x +end

  • +"""

  • mean!(f, r, A)

  • +Apply f to each element of A, and compute the mean over the singleton dimensions of r, and write the results to r. +""" +function Base.mean!(f::Function, r::AbstractArray{T}, a::AbstractArray) where {T<:Number}

is there a reason why the element type of a is not fixed?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/XtractOpen/Meganet.jl/pull/48#pullrequestreview-95631522, or mute the thread https://github.com/notifications/unsubscribe-auth/ATplBcCDRsoy-_cUFdyvh6faqJzz_xXfks5tTfGWgaJpZM4R9reu .