make Marshaler and Loadable work together

[okhowang]

I use gocache as a cachable mysql wrapper.

    bc, err := bigcache.NewBigCache(bigcache.DefaultConfig(time.Minute))
    if err != nil {
        panic(err)
    }
    return cache.NewLoadable(func(key interface{}) (interface{}, error) {
        var data DataType
        err := db.Get(&data, "select * from some_table where some_id = ?", key)
        return data, err
    }, cache.New(store.NewBigcache(bc, nil)))

but there is panic as below

goroutine 40 [running]:
github.com/eko/gocache/store.(*BigcacheStore).Set(0xc022c311d0, 0x1266220, 0xc0005b0400, 0x13bd900, 0xc0266114a0, 0x0, 0xc000084401, 0xc0005b0400)
    E:/go/pkg/mod/github.com/eko/gocache@v1.1.1/store/bigcache.go:68 +0x1a5
github.com/eko/gocache/codec.(*Codec).Set(0xc022c311e8, 0x1266220, 0xc0005b0400, 0x13bd900, 0xc0266114a0, 0x0, 0x1, 0x101)
    E:/go/pkg/mod/github.com/eko/gocache@v1.1.1/codec/codec.go:66 +0x70
github.com/eko/gocache/cache.(*Cache).Set(0xc0003b0030, 0x1266220, 0xc0002105a0, 0x13bd900, 0xc0266114a0, 0x0, 0x0, 0x0)
    E:/go/pkg/mod/github.com/eko/gocache@v1.1.1/cache/cache.go:45 +0xb1
github.com/eko/gocache/cache.(*LoadableCache).Set(...)
    E:/go/pkg/mod/github.com/eko/gocache@v1.1.1/cache/loadable.go:68
github.com/eko/gocache/cache.(*LoadableCache).setter(0xc000090020)
    E:/go/pkg/mod/github.com/eko/gocache@v1.1.1/cache/loadable.go:41 +0x93
created by github.com/eko/gocache/cache.NewLoadable
    E:/go/pkg/mod/github.com/eko/gocache@v1.1.1/cache/loadable.go:34 +0xbf

Loadable set value directly to bigcache which accept []byte only.

make Marshaler as CacheInterface may be ok.

    bc, err := bigcache.NewBigCache(bigcache.DefaultConfig(time.Minute))
    if err != nil {
        panic(err)
    }
    return cache.NewLoadable(func(key interface{}) (interface{}, error) {
        var data DataType
        err := db.Get(&data, "select * from some_table where some_id = ?", key)
        return data, err
    }, Marshaler.New(cache.New(store.NewBigcache(bc, nil))))

[adabour-thermeon]

I faced a similar problem when trying to use Marshal with Loadable for Redis, they both do not work together.

I exteded Loadable with Marshal, but I'm not sure if it is the correct way to do that.

package cache

import (
    "github.com/vmihailenco/msgpack"
)

type Marshaler struct {
    LoadableCache
}

func NewMarshaler(loadFunc loadFunction, cache CacheInterface) *Marshaler {
    m := Marshaler{}
    loadable := &LoadableCache{
        loadFunc: loadFunc,
        cache:    cache,
    }
    m.LoadableCache = *loadable
    m.setChannel = make(chan *loadableKeyValue, 10000)
    go m.setter()
    return &m
}

func (c *Marshaler) setter() {
    for item := range c.setChannel {
        bytes, err := msgpack.Marshal(item.value)
        if err != nil {
            continue
        }
        c.Set(item.key, bytes, nil)
    }
}

// Get obtains a value from cache and unmarshal value with given object
func (m *Marshaler) Get(key interface{}, returnObj interface{}) (interface{}, error) {
    result, err := m.cache.Get(key)
    if err != nil {
        // Unable to find in cache, try to load it from load function
        result, err = m.loadFunc(key)
        if err != nil {
            return nil, err
        }
        // Then, put it back in cache
        m.setChannel <- &loadableKeyValue{key, result}
        return result, nil
    }

    switch v := result.(type) {
    case []byte:
        err = msgpack.Unmarshal(v, returnObj)
    case string:
        err = msgpack.Unmarshal([]byte(v), returnObj)
    }

    if err != nil {
        return nil, err
    }

    return returnObj, nil
}

[okhowang]

my solution is make load function return a []byte and wrap loadable with marshaler

    bc, err := bigcache.NewBigCache(bigcache.DefaultConfig(time.Minute))
    if err != nil {
        panic(err)
    }
    return marshaler.New(cache.NewLoadable(func(key interface{}) (interface{}, error) {
        value, err := loadFunction(key)
        if err != nil {
            return nil, err
        }
        return msgpack.Marshal(value)
    }, cache.New(store.NewBigcache(bc, nil)))), nil

but it's depend on that marshaler use msgpack

[eko]

Now that Loadable cache support generics, I think we should make Marshaler support for generics too.

[jasonlimantoro]

my solution is make load function return a []byte

nice workaround. Although this couples the loader function with the marshaling process, which isn't ideal. Ideally, loader function should just return (struct, error), nothing less nothing more. @okhowang

Now that Loadable cache support generics, I think we should make Marshaler support for generics too.

Any timeline we can expect this to be supported? Also can we support json marshaler? Though we can write our own marshaler, I still think jsonmarshaler is a sensible battery-included solution. @eko

[bohdand-weka]

I had to write my own Marshaler, which works over store, not over cache interface. It should fit better also in this case