Backend Implementation

[falquaddoomi]

This PR includes an implementation of /neighbors that comes in around 25 seconds on an e2-standard-8 (8 vCPU, 32GB RAM) with an SSD.

Configuration Settings

The container uses the following environment variables to tweak how it works:

• USE_MEMMAP: (Boolean) Employs gensim/numpy's memory-mapping to share as much data as possible between processes.
• MATERIALIZE_MODELS: (Boolean) Loads all models into memory, rather than loading them one at a time as they're queried.
• WARM_CACHE: (Boolean) Preloads the models at server boot, instead of at first web request.
• PARALLELIZE_QUERY: (Boolean) Parallelizes queries for different years using joblib.
• PARALLEL_POOLS: (Integer) If parallelization is enabled, how many parallel workers to run.
• PARALLEL_BACKEND: (Enumeration: "threading", "loky", "multiprocessing") Which joblib.Parallel backend to use.

Profiling

The ./server/profiling folder contains a script run_profiling.sh that can be used to compare different configurations of the above settings. The script runs all the profiles declared at the beginning of the script against the API VM. It does the following:

1. Polls/ to ensure the site is up (e.g., because WARM_CACHE can delay when the server first becomes available by up to 5 mintues)
2. Runs three queries in serial for /neighbors?tok=pandemic
3. Records the profile time, initial warmup time, and the latency and HTTP response codes for each of the profiles to profiling_results__<TIMESTAMP>.csv.

To-Dos, Notes

• The profiling script should be updated to compare concurrent (and also staggered) requests, since that will be a large source of load, especially when not using memory-mapped IO or materialized models. Right now there's just a single client running three queries, one after the other, which isn't exactly realistic.
• The server still needs to be right-sized to optimize performance vs. what we're willing to pay. It's likely that it's both overprovisioned and underutilized at the moment...
• More investigation is needed into the tradeoffs in performance and memory footprint between memory-mapped IO and giving each parallel worker its own data, threading vs. processes, and the number of parallel workers.
• Ideally we'd do the profiling experiments on another staging server and leave the API server somewhat stable, so the frontend can be coded against it. (It's not a hard change, but we'll need to pay for another server.)

[falquaddoomi]

FYI, I added the Redis caching to this PR, since it's kind of essential to getting good performance from the server. I'll be adding the cache populating script in a separate PR.

[netlify[bot]]

✔️ Deploy Preview for word-lapse canceled.

🔨 Explore the source changes: e79cf307bf3f6f79d8624de7df5009a4ce2b6260

🔍 Inspect the deploy log: https://app.netlify.com/sites/word-lapse/deploys/6216b76f4b3fbf0007c1756e

[falquaddoomi]

Hey @danich1, I realize this is a huge PR now, my apologies. Feel free to review it now, and let me know if you have any questions.

[falquaddoomi]

FYI, I just ran black on the backend code as suggested. I'm going to merge shortly, then start submitting PRs for the other outstanding issues.