usdigitalresponse / arpa-reporter

Web app to aid government agencies in reporting on ARPA grants.
Apache License 2.0
3 stars 0 forks source link

Memoize filesystem reads for uploads (within request) #497

Closed ajhyndman closed 2 years ago

ajhyndman commented 2 years ago

Add a memoization layer to upload reads. This ensures that the same upload is not read from disk more than once within a single request.

The audit report generation process could end up reading the same upload from disk two or more times. Rather than try to complicate the logic for report generation, this change caches audit report reads.

To avoid leaking memory, this cache is only preserved on the express request object. This allows the node process to garbage collect cached uploads after the request has resolved. This will be very important in a multi-tenant deployment.