These FreeLists were responsible for 5.4MiB of allocations per query.
This is an excessive amount for systems that may have hundreds or
thousands of queries in flight at any particular time. They are now
shrunk to 82KiB per query.
The sizes were determined by some internal experimentation; they are the
smallest sizes that didn't cause reallocations on a typical NER task
(for us). YMMV for other tasks.
These FreeLists were responsible for 5.4MiB of allocations per query. This is an excessive amount for systems that may have hundreds or thousands of queries in flight at any particular time. They are now shrunk to 82KiB per query.
The sizes were determined by some internal experimentation; they are the smallest sizes that didn't cause reallocations on a typical NER task (for us). YMMV for other tasks.