ThummeTo / FMIFlux.jl

FMIFlux.jl is a free-to-use software library for the Julia programming language, which offers the ability to place FMUs (fmi-standard.org) everywhere inside of your ML topologies and still keep the resulting model trainable with a standard (or custom) FluxML training process.
MIT License
57 stars 15 forks source link

A dedicated Cache-Retrieve layer which creates a vector of pairs of the used vector and the cached is missing. #101

Closed juguma closed 8 months ago

juguma commented 1 year ago

For the extended ScaleSum (cf. #100) it is helpful to have a matching Cache-Retrieve layer, which prepares value pairs (2-Vectors) of the entered "dx"-Vector and the values of the cache at the specified indices. The current version creates a vector with cached values only before or after the entered "dx"-vector, which even with a resphape cannot be transformed into the expected (and handy) format of a Vector of 2-Vectors. I propose to define a new Cache-Retrieve-Layer-Variant, as follows:

cache = CacheLayer()
v7 = [1.0,2.0,3.0,4.0,5.0,6.0,7.0]
cv7 = cache(v7)
cRL = CacheRetrieveLayer(cache)

idxs = vcat(1,3,5:6)#

#->I would like to have:
pairVec = [[v7[1],cRL(1)],[v7[3],cRL(3)],[v7[5],cRL(5)],[v7[6],cRL(6)]]

#->proposed new PairWithCacheRetrieve layer to simplify this line:
struct PairWithCacheRetrieve
    cacheLayer::CacheLayer

    function PairWithCacheRetrieve(cacheLayer::CacheLayer)
        inst = new(cacheLayer)
        return inst
    end
end
export PairWithCacheRetrieve
function (l::PairWithCacheRetrieve)(idx,x)
    #for all idx writes cache[idx],x[idx] <-i.e. length of x must be equal to lenght of idx 
    tid = Threads.threadid()
    retVal = typeof(x)[]
    sizehint!(retVal, length(idx))
    for i in 1:lastindex(idx)
        push!(retVal, [l.cacheLayer.cache[tid][idx[i]], x[i]])
    end
   return retVal
end

#->usage:
cRPairL = PairWithCacheRetrieve(cache)
y = cRPairL(idxs, v7[idxs])
y == pairVec

#->and then this output can be used with the "vectorized" scalesum ;-)
scsum(y)

I think the sizehint and push could be replaced by the correct allocation of the retVal Vector of 2-Vectors which is more performant(!?), but I haven't figured out how to allocate it correctly, yet.

juguma commented 8 months ago

The current implementation of the ScaleSum (which typically acts on pairs of cached and modified values) might not be intuitive and the jumping back and forth in the x-Vector is probably less efficient than working with a preordered Vector of 2-Vectors (cf. #100 where we made some tests). However, the new definition of the CacheRetrieveLayer allows for a more flexible sequence concatenation of cached values and non-cached values, which in combination with the ScaleSum leads to the required output. Hence I close this ticket.