Memory table reloads are a pain, and require scheduling using cron jobs or equivalents that tend to be system-specific and hard to support in a general way. We should be able to add support for automatically scheduling these jobs on a few platforms (debian, OS X) but even then the cron tasks will fail if the user doesn't want to be launching MySQL automatically with every launch.
So concurrently, we should allow (and perhaps even make the default) creating bookworms that store their fast lookup tables on disk rather than in memory. With a command line argument to the new interface, this should much easier to implement than in previous versions.
The downside of disk tables is that they are slower than memory tables, and probably inappropriate for bookworms of more than several hundred thousand documents. For the smaller ones, though, they will improve reliability and reduce memory use.
All this will take is searching for every occurrence of MEMORY in variableSet.py and CreateDatabase.py (and maybe only the former file), and replacing it with a substitution that returns either MEMORY or MYISAM depending on a flag passed to the command line arguments.
bmschmidt
commented
7 years ago
Memory table reloads are a pain, and require scheduling using cron jobs or equivalents that tend to be system-specific and hard to support in a general way. We should be able to add support for automatically scheduling these jobs on a few platforms (debian, OS X) but even then the cron tasks will fail if the user doesn't want to be launching MySQL automatically with every launch.
So concurrently, we should allow (and perhaps even make the default) creating bookworms that store their fast lookup tables on disk rather than in memory. With a command line argument to the new interface, this should much easier to implement than in previous versions.
The downside of disk tables is that they are slower than memory tables, and probably inappropriate for bookworms of more than several hundred thousand documents. For the smaller ones, though, they will improve reliability and reduce memory use.
All this will take is searching for every occurrence of
MEMORYinvariableSet.pyandCreateDatabase.py(and maybe only the former file), and replacing it with a substitution that returns eitherMEMORYorMYISAMdepending on a flag passed to the command line arguments.