Add second option for CACHALOT_CACHABLE_TABLES in a db router

[Andrew-Chen-Wang]

This issue is linked to the "Integrating Architect with Cachalot" project. This project is meant to help integrate dynamic table caching.

• [ ] Add functionality to cache tables in a callable, or simply integrate it with a Django router.
• [ ] Write documentation for how this can be achieved with architect

[Andrew-Chen-Wang]

This unfortunately isn't really solved with #187. The proposal was to add a wildcard but I thought that degrade performance. I'll run a benchmark against both options, but I'm pretty sure it's not going to go well.

The problem with the wildcard solution is how are we actually supposed to filter out which tables go into cachable and uncachable. I suppose we could do that in the settings beforehand (i.e. load all the tables, then do a final check by filtering out all those wildcards). Besides that, I think the callable would degrade performance too much and thus a wildcard would in fact be a better solution.

[Andrew-Chen-Wang]

We can use something like fnmatch to quickly do wildcard matches, but this isn't as versatile as regex for sure. Again, it's down to that performance issue. Key factors:

• Memory (since you're loading a huge cache I suppose)
• Efficiency via overengineering (Python is fast enough to not really care, but considering the ORM is used all the time, every millisecond counts especially with latency being the major factor, backend being fast is key).

[Andrew-Chen-Wang]

Another strategy is to 1. use fnmatch and 2. allow regex OBJECTS (not string values). It's as simple as doing isinstance(searchable, str) to determine whether to use fnmatch or else we know that the next object must be a regex object.

But should we compile all regex values? Yes without a doubt... maybe we can combine two settings objects as well so that people CAN use just strings without importing regex. The thing is, doesn't regex have some global value? Ok so maybe we do need the user to first compile their regex values first. I think based on a name like CACHALOT_REGEX_CACHABLE_TABLES would peak people's curiosity but would also lead them to be confused enough to check the documentation. By checking the docs, devs will know what we need (specifically, compiled regex).

We have to use compiled regex objects that the user does themselves because we search those tables quite frequently... like a lot a lot. And the reason strings values themselves don't work is because people like to add flags too.