How to implement custom stores, for example redis?

[mirusky]

How to implement custom stores, for example redis?

I've tried to use the package with redis but seems that I can't decode/encode the values properly:

panic: reflect: call of reflect.Value.Elem on map Value, it's occurs because I'm Mashal/Unmarshal values, but idk how can I safely save it on redis. Looks like here I have src as map not as a proper way to reflect back to dst

package cache

import (
    "encoding/json"
    "time"

    "github.com/go-gorm/caches/v2"
    "github.com/gofiber/fiber/v2"
    "github.com/gofiber/storage/redis"
    "gorm.io/gorm"

    "core/env"
)

type customCacher struct {
    storage fiber.Storage
}

func newCustomCacher(config env.Config) caches.Cacher {
    storage := redis.New(redis.Config{
        URL:      config.RedisURL,
        Database: 3,
    })

    return &customCacher{
        storage: storage,
    }

}

func (c customCacher) Get(key string) *caches.Query {
    val, err := c.storage.Get(key)
    if val == nil || err != nil {
        return nil
    }

    var query caches.Query
    if err = json.Unmarshal(val, &query); err != nil {
        return nil
    }

    return &query
}

func (c customCacher) Store(key string, val *caches.Query) error {

    b, err := json.Marshal(val)
    if err != nil {
        return err
    }

    return c.storage.Set(key, b, 6*time.Hour)
}

func NewGormCacherPlugin(config env.Config) gorm.Plugin {

    return &caches.Caches{
        Conf: &caches.Config{
            Easer:  true,
            Cacher: newCustomCacher(config),
        },
    }
}

The document you expected this should be explained

Custom cacher, with different stores than memory.

Expected answer

A consise way to store in another place the informations

[mirusky]

@ktsivkov can you provide some help?

[ktsivkov]

Hey @mirusky I will have a look at it later. My initial thought/idea is to use some way of Marsh unmarshal of the data passed and retrieved to and from the Cacher. Three problems need to be addressed -

• Detect if we are having primitive type, or slice, or map.
• If slice or map - is there a model structure inside.
• • If so, which type is it, and how to marshal unmarshal.
• • What happens with the gorm.Model embedded structure, how do we handle it.

Things I would test/try: If we use json marshal/unmarshal we need to identify the correct unmarshalling destination, and apply json struct tags.

It surely needs some deeper investigation and edge case identification.

[mirusky]

MsgPack and JSON, are getting the wrong value in Get method. Since the caches.Query has Dest as interface{}, the decoder doesn't know what are the proper way to decode / what's the concrete type to decode. So almost all the cases it's transformed into a map or a slice of maps.

I've tested with gob, but I need to call gob.Register to all possible types to make it work. Since it's a big repo it's impracticable.

I don't have sure, but can we have the Model it's trying to solve in the Get method? Like:

Get(key string, model interface{})

So in that way we could ensure that I have the correct type to Unmarshal.

Or in SetPointedValue do a full reflection of the value?

[mirusky]

Also I'll try to create a small example and attach here.

[ktsivkov]

Hey @mirusky , please check this PR I just created, if it works for your usecase we can merge and release it. https://github.com/go-gorm/caches/pull/7

In order to use the Marshal & Unmarshal call the receiver functions of the query. If you need any more clarifications let me know.

[ktsivkov]

@mirusky As for your comment suggesting the Model Interface, it could actually work! Will check later today, if you figure it out faster you might create a PR for me to review!

Cheers!

[Ahbrown41]

Any luck with this one? Would be great to have a solution on how to do it..

[ktsivkov]

@mirusky @Ahbrown41 Please check https://github.com/go-gorm/caches/releases/tag/v3.0.0 in the readme I have added examples of how to implement Redis caching. I tried to test it in different ways but it'd still be a good idea to use with some caution. If any critical bugs are found please open issues, contributions are welcomed as well. For quite some time I was occupied and did not have the time to resolve it.

[mirusky]

Looking promising, I've ended up creating a cache in another way ( getting the query as key, and result as value ).

But as you implemented redis here, I'll have a try.