Investigate long load times for both SenNet/HuBMAP dev and prod

[maxsibilla]

For select users both consortia and both dev and prod versions of the site take over a minute to load. Things to potentially investigate:

• Are entities being cached properly
• Are certain cypher queries stalling/taking a long time (potentially due to broken provenance chains)

[maxsibilla]

After investigating this it seems that one of the longer tasks inside /data-status is the files_exist check (still pretty negligible though). The larger issue is that on the HuBMAP Ingest API it is set up to intermittently run this function and cache the results to a REDIS cluster. I have two theories:

• The REDIS cluster does not have enough RAM allocated and the cache is being flushed when it runs out of memory
• There is another library being used (rq https://github.com/hubmapconsortium/ingest-api/blob/3f6430ef75a57251e2b603bd9f760d7be0afb35d/src/app_utils/task_queue/__init__.py#L2) could this be affected the cache at all?