Non-Allocating ACE-Lux Networks

[cortner]

@CheukHinHoJerry : This is an idea how to get rid of the nasty allocation you have in sparsesymmprodand all the other layers too. Im using incorrect names here because I don't remember the correct ones, but the gist is hopefully clear:

struct P4ML_LuxLayer 
      # all the other stuff, then add the following field: 
     release_input::Bool
end

function (l::P4MLLuxLayer)(X, ps, st) 
     out = acquire!(st, ....)
     evaluate!(out, l.basis, X, ps, st.temp)
     if l.release_input
          release!(X)
     end 
    return out 
end

Basically this will enable you to release each computed basis once it is no longer needed back into the pool, and this allocation should then go away. What do you think?

[CheukHinHoJerry]

Thank you. I will think this through and do this together with the trainable layer interface that we mentioned.