Idea: Use an LLM agent for polishing the story

[lfoppiano]

We could use an LLM agent for fetching the story and polish it, at least would be interesting to know how well it could behave in comparison with our current approach

[andythean]

I did some trials using an LLM for summaristion/style convesion (podcastification) of the feeds and stories and topic clustering according to user-defined topics of interest. For me, the most promising avenues were style comversion and topic clustering. One could cut out the user-interaction completely by delivering a curated collection of news in an easy-to-digfest style e.g. create a custoim, up-to-the-minute 'podcast' . NotebookLM demonstrates the feasibility of podcastification, the UI for user-defined topic defintioin would need working out: could be learned from history of interactions with current code ? Could be defined expliclty by user e.g. via speech recognition ? Based on static topic menu ? Based on dynamic topic menu extracted on the fly ?

[lfoppiano]

My idea was simpler, just a way to remove unlikely pieces of text that that might be ending up in the story (other stories, ads, etc..).

IMHO The LLM might be useful on this, as they are likely to identify low probability next tokens.

Since there is a plan to improve the BERT model this could be a parallel experiment to evaluate.

[andythean]

Okay, I see. Along this line, the previous trials used the BERT Next Sentence Prediction probabilities (using BertForNextSentencePrediction form the transformers library), as features for classification. Eventually they were removed becuase the ablation trials showed they weren't high value despite being high computational cost. Having said this, in theroy they should add value and I had some dounts about whether the NSP probs were really what I thougfht they were due to what I read on some blogs. Newer models might improve perfomance, but the extra computatioal cost will remain something to consider...