search

jogden17 / CapstoneProject

0 stars 0 forks source link

Minimize summarization time #10

Closed jogden17 closed 4 months ago

jogden17 commented 4 months ago

Reduce summarization time to allow for the option to allow users to adjust the summary length

jogden17 commented 4 months ago

Implemented library to allow phi-3 model to use metal performance shaders for improved computation speed. This comes at the cost of greatly increasing the RAM used by the model, which was reduced by clearing pytorch cache between articles. Time to summarize a single headline taken from 15-30 minutes down to 1-5 minutes.