Refactor large all content csv donwloads

mediacloud / web-search

Code that drives the public web-based tools for the Media Cloud Online News Archive and Directory.

https://search.mediacloud.org

Apache License 2.0

9 stars 15 forks source link

Refactor large all content csv donwloads #790

Open Evan-Leon opened 1 month ago

Evan-Leon commented 1 month ago

When attempting to download large csvs for all content, it is giving an error banner of Screen Shot 2024-09-18 at 1 09 34 PM investigate TotalAttentionEmailModal for fix

-also views.py, looks to be iterating still over all queries

Evan-Leon commented 1 month ago

Additionally there are limits hard coded in front end TotalAttentionEmailModal and back end in views, would be better to have these limits as Env Vars to be configured via web search config file

philbudne commented 1 week ago

On last comment: ALL_URLS_CSV_EMAIL_MAX and ALL_URLS_CSV_EMAIL_MIN config vars now available in browser as document.settings.allUrlsCsvEmailMin and document.settings.allUrlsCsvEmailMax