uh-dcm / news-article-collection-container

News article collection tool for research
MIT License
2 stars 1 forks source link

Streamed, resource efficient downloads and wordcloud saving improvement #222

Closed ayriainen closed 3 weeks ago

ayriainen commented 4 weeks ago

Original downloads struggled with Rahti resource limits, for example peak memory usage becoming many times that of the size of the download file. By streaming and more conservative memory usage, JSON and CSV files are able to be sent over to the user immediately in chunks as they are converted from the database data.

Word cloud saving was also improved slightly with the zoom and navigation buttons now having an effect on the end result. This mattered on some screens where zooming out is necessary to view the word cloud properly, as otherwise the save function simply saved the unzoomed version.

A screenshot of the dashboard was also added to readme to help new users understand what this repository is about.