SIS-Team-24 / 2023-SIS-Team-24

Using Natural Language Processing, Text Insights examines and provides insights to text.
1 stars 0 forks source link

Performance Issues #188

Closed henrygoodman closed 1 year ago

henrygoodman commented 1 year ago

Summary time seems to grow exponentially with the input size, even with extractive summary.

Running the summary with 88k words bricked my computer, and I saw the RAM usage by Python exceed 15GB (huge)

Either research into potential performance optimizations (by anyone who has done this type of thing before) or set an upper limit on words (words is better because it saves us time having to calculate num tokens)