search

bitcoinsearch / summarizer

Generation of easily accessible summarization feeds for public consumption
MIT License
5 stars 5 forks source link

Summarization of Bitcoin Transcripts Data #52

Closed urvishp80 closed 7 months ago

urvishp80 commented 7 months ago
kouloumos commented 7 months ago

This is great, @urvishp80! Next time you're working with Bitcoin Transcripts, please feel free to request my review—I believe I can offer some valuable insights beforehand.

Some thoughts/questions:

  1. If we already have a document in Elasticsearch for each transcript, why duplicate it here? I understand the need for using summaries (TLDR) as we do for other sources, but is duplicating the entire document necessary?
  2. If our aim is to summarize transcripts, this repository might not be the ideal place for storing those summaries. Ideally, we want to display the summaries on btctranscripts.com. We already incorporate some summaries within the metadata of each transcript in the bitcointranscripts repo. A more efficient workflow would be to use the logic here to generate summaries for transcripts that lack them and then commit these summaries directly to the bitcointranscripts repo. Subsequently, a cron job could verify whether each transcript document in Elasticsearch includes a summary and, if not, extract the summary from the metadata and append it to the document.
  3. Based on Issue #63, this logic will also generate summaries for AI-created transcripts before they are finalized. I’m not suggesting this is undesirable—it all hinges on their quality. If they are of high quality, these summaries could help reviewers decide which transcripts to claim and review.
  4. I’m experimenting with various prompts for summarizing as well. It might be beneficial for us to exchange notes.
  5. A minor point: We should probably start thinking about renaming this repository, as it has evolved to function more as a "Summarizer" than merely "mailing-list-summaries."