openfga / openfga

A high performance and flexible authorization/permission engine built for developers and inspired by Google Zanzibar
https://openfga.dev
Apache License 2.0
2.7k stars 181 forks source link

Solve the "new ennemy" problem using zookie's token #1777

Open Scaranova opened 1 month ago

Scaranova commented 1 month ago

Checklist

Describe the problem you'd like to have solved

I'm currently doing a POC to compare SpiceDB and OpenFGA. However, I noticed that the SpiceDB solution mentions zookie tokens (used by Google Zanzibar) in order to improve performance. And I was wondering if there would be an equivalent in OpenFGA with the authorization model system for example? Or if it is not yet implemented/will be implemented?

Here is Google's paper with the explanation of this token from page 3: https://storage.googleapis.com/pub-tools-public-publication-data/pdf/0749e1e54ded70f54e1f646cd440a5a523c69164.pdf

Describe the ideal solution

Implement (if there is no equivalent yet) a zookie token system directly or using the authorization model system

Alternatives and current workarounds

No response

Additional context

No response

aaguiarz commented 1 month ago

Hi @Scaranova

OpenFGA does not currently support zookies. If you enable the cache, then are not protected from the New Enemy problem.

We are wrapping up the implementation of an additional feature to OpenFGA queries where you can specify a consistency parameter, that will initially have two values: 'minimize latency' and 'higher consistency'. If higher consistency is specified, OpenFGA will skip the cache.

We plan to also start returning a consistency token (zookie) from writes that you can later send to query endpoints. Depending on the token, OpenFGA will decide if it uses the cache or reads directly from the database.

Note that when using a consistency token, you need to store it on your database tables to then send it. This adds complexity to the implementation in two ways:

Also, in a lot of cases, you probably already have a 'last modified' field database tables. If you can use that field that already exists, the cache TTL, and decide if you do a consistent read or use the cache. If the cache TTL is 10s, you'll make a consistent read only if the current time + 10s < last modified date. That way you avoid changing the schema.

Let me know if this helps.

Scaranova commented 1 month ago

Yes, thanks a lot @aaguiarz! I can't wait to see how OpenFGA will evolves in this direction!

Scaranova commented 1 month ago

Hello @aaguiarz , Im sorry but i have another question, I am currently doing load tests on OpenFGA and SpiceDB to compare the 2 solutions, but I noticed that SpiceDB stores everything in RAM, while OpenFGA will store between 50 and 200MB it seems to me in the ram before sending to the database.Which can create a rather large performance gap between the two (Spice DB is 2 to 4 times faster) Would it be possible to increase the size of this cache? I can't do it by modifying the docker compose file. Thank you in advance for your response!

aaguiarz commented 1 month ago

You can configure the cache size using the maxCacheSize setting.

https://github.com/openfga/openfga/blob/cea5aa1034e3aab9f3c794f7a940b82c01caf085/.config-schema.json#L186

Scaranova commented 1 month ago

Thanks @aaguiarz!

aaguiarz commented 1 month ago

Feel free to reach out directly to me at andres.aguiar@openfga.dev if you need help with the load test