vektah / dataloaden

go generate based DataLoader
MIT License
530 stars 79 forks source link

Can disable caching? #52

Open KingWu opened 4 years ago

duong-se commented 3 years ago

Just have same question, could I disabled caching because it'll cause an issue when we change data

UPDATED: You can use clear cache by this method

func (r *shipmentResolver) Customer(ctx context.Context, obj *model.Shipment) (*model.Customer, error) {
    customer, err := dataloader.For(ctx).CustomerByID.Load(convertedID)
        // Clear after loaded it
    dataloader.For(ctx).CustomerByID.Clear(convertedID)
    if err != nil {
        // Handle Error
    }
    return customer, nil
}
frederikhors commented 3 years ago

@tanphamhaiduong can I ask you why Clear() is needed?

duong-se commented 3 years ago

@tanphamhaiduong can I ask you why Clear() is needed?

Because some entity i need to update frequent, so clear cache to make sure it return correct value for the user

frederikhors commented 3 years ago

Ok. But I still don't get it. Are you updating that entity frequently during that call?

sandorvasas commented 2 years ago

Need to be able to disable caching. Causing issues for us immediately right after deploy. User changed its name, but the old user name is getting loaded. Not good. I would don't need caching at all. It's causing more problems than it solves.

TroyKomodo commented 2 years ago

I am unsure why caching is even implemented at all. Caching should and always should be controlled by the developer. If i wanted to add caching I would have implemented it in the data loader step, when i fetch the data from the database, ie checking redis or a in memory cache. I am likely going to have to fork to remove this functionality and it this feature should be seriously reconsidered.

TuringJest commented 2 years ago

The caching is intended to work per request and not to be long-lived. To make sure this works properly you have to return a new dataloader for every request .

ldrs = Loaders{
Todos:  newTodoLoader(),

func newTodoLoader() *TodoLoader {
    return &TodoLoader{
        wait:     wait,
        maxBatch: 1000,
        fetch: func(keys []ID) ([]*Todo, []error) {
                         // ... retrieve data
            return todos, errors
        },
    }
}

Also make sure you are using the context as in the example folder.

TroyKomodo commented 2 years ago

@TuringJest this is not correct.

  1. Why would you want dataloading to be scoped to a single request when you can scope it to the entire application?
  2. If I re-query the dataloader you would assume that the function is rerunning the database queries, if I wanted to implement caching, at a request or app scope, I would have done it in my own code. Limiting the dataloader to be used in this way is quite strange.
TuringJest commented 2 years ago

The caching that happens on a request level doesn't stop you to use your own global caching solution to retrieve data inside the dataloader.

It just ensures that the data returned in one request is consistent in all cases and not re-queried if requested multiple times across one request.

But you can check https://github.com/graph-gophers/dataloader which offers to disable caching completely.
If you want to use this with gqlgen you can also find an example in the documentation of the master-branch.

TroyKomodo commented 2 years ago

@TuringJest I think you miss-understood.

There is no way to have a global data loader without caching, your proposed solution to get around the caching of recreating it per request is not global. I fixed this problem by forking it and removing the caching aspects. Caching here is stupid, because every application has different requirements and different needs, perhaps a better way to approach this is to allow a ARGV flag which disables the caching code generation, although I didn't do this because I don't really need caching at all.

This library is better than the graph-gophers lib because this is a code generator which means all the logic is run and confirmed at compile time (which is better) than running and checking logic at runtime (which is error prone). They both do data loading but generally being able to force types at compile time is far superior than forcing them at runtime.