search

rdaum / moor

A system for building shared, programmable, online spaces. Compatible with LambdaMOO.
GNU General Public License v3.0
160 stars 8 forks source link

Support quota / E_QUOTA #7

Closed rdaum closed 9 months ago

rdaum commented 1 year ago

Requires at least estimating object sizes and tracking them per user / owner. Need some hooks on the WorldState interface.

rdaum commented 1 year ago

This work is basically:

  1. Add to the WorldState trait a function to query quota per object owner.
  2. Likewise, add the ability to ask WorldState for the size of a specific object.
  3. Fixup stuff in values so that Var and VerbDef etc have reasonably efficient "tell me your size in bytes" functions.
  4. The trickier part: Actually implement the functions in 1. and 2. for the rocksdb backend. Would mean tracking byte sizes for all the different column families that bits of objects are stored in, since an object is never stored as a whole chunk.
  5. Implement the quota checks and tracking in create and recycle
  6. Fix up the byte-size relates builtins in bf_values.rs to use the stuff from 3.
pppery commented 1 year ago

LambdaMOO's quota implementation, by the way, is almost entirely done in-database, only relying on an object_size builtin. The quota-handling code in built-in create and recycle is a relic of way back when when object quota rather than byte quota was used.

And you could even not do that and reactivate $quota_utils:object_bytes(old) and LambdaMOO's quota system would almost just work.

rdaum commented 1 year ago

Thanks, I keep meaning to dig into it in earnest, but part of the problem is I really don't like the way the bytes quota stuff ends up putting some annoying requirements into the DB layer that are rather awkward to implement. Knowing how many bytes an object uses up is a bit tricky in the context of systems built with paged datastructures and the like where the packing of a given page depends on branching factors and etc and what not, and in my current DB impl all the pieces that make up what an object is are scattered across multiple "column families" etc. I'd prefer to avoid having to scan all of them to go look it up for a given object, so I will probably have to add specific summary tables, cache them etc.

rdaum commented 9 months ago

So yeah, as per @pppery's comment, I'm hoping/assuming I do not need to implement the "old school" create/recycle object-count based quota at all, and as I have a working object_bytes now, I think that's all that's needed.

I haven't done any rigorous testing with a core, but if anything comes up I'll re-open this ticket. For now I'll consider this done.

Remaining related work is implementing db_disk_size in some form, but that's not about quota, really.