a possible bug in code: title_chunks.append(papers[start:])

outline_writer.py

    def chunking(self, papers, titles, chunk_size = 14000):
        paper_chunks, title_chunks = [], []
...

        paper_chunks.append(papers[start:])
        title_chunks.append(papers[start:])   # <<-------  should this be titles[start:] instead?

        return paper_chunks, title_chunks

The log I generated confirms that the title is the same as the content in the generated paper list:

2024-07-20 22:02:18,125 - INFO - Starting outline generation for topic: generating code summary using large language model, using ref_num=100, chunk_size=30000, section_num=3
2024-07-20 22:02:19,136 - INFO - Retrieved 100 relevant paper IDs from the database.
2024-07-20 22:02:28,219 - INFO - chunking resuls: 1 abstract chunks 1 title chunks.
2024-07-20 22:02:28,219 - INFO - Generating rough outlines for 1 chunks.
2024-07-20 22:02:28,253 - INFO - entering batch chat api model
2024-07-20 22:02:28,253 - INFO - entering api_model. batch_chat(), max threads= 5...
2024-07-20 22:02:28,253 - INFO - Input 1 text_batch:
2024-07-20 22:02:28,253 - INFO - Prompt 1: 
You wants to write a overall and comprehensive academic survey about "generating code summary using large language model".
You are provided with a list of papers related to the topic below:
---
--- 
paper_title: Large language models pre-trained for code generation can generate
high-quality short code but often struggle with generating coherent long code
and understanding higher-level or system-level specifications. This issue is
also observed in language modeling for long text generation, and one proposed
solution is the use of a latent stochastic process. This approach involves
generating a document plan and then producing text that is consistent with it.
  In this study, we investigate whether this technique can be applied to code
generation to improve coherence. We base our proposed encoder and decoder on
the pre-trained GPT-2 based CodeParrot model and utilize the APPS dataset for
training. We evaluate our results using the HumanEval benchmark and observe
that the modified Time Control model performs similarly to CodeParrot on this
evaluation.

paper_content:

Large language models pre-trained for code generation can generate 
high-quality short code but often struggle with generating coherent long code
and understanding higher-level or system-level specifications. This issue is
also observed in language modeling for long text generation, and one proposed
solution is the use of a latent stochastic process. This approach involves
generating a document plan and then producing text that is consistent with it.
  In this study, we investigate whether this technique can be applied to code
generation to improve coherence. We base our proposed encoder and decoder on
the pre-trained GPT-2 based CodeParrot model and utilize the APPS dataset for
training. We evaluate our results using the HumanEval benchmark and observe
that the modified Time Control model performs similarly to CodeParrot on this
evaluation.
---

AutoSurveys / AutoSurvey

a possible bug in code: title_chunks.append(papers[start:]) #6