Disabling the 2nd summary

[s11i-code]

Hi!

It would be nice if one could opt out of the 2nd summary. (Or maybe there's a way, I just didn't find it.) For my use case, the 2nd summary step (the summary of summaries) made the result a little too short, I believe I would have only needed one level of summarization to get to the length I needed. Really nice tool, nevertheless.

[dimays]

Great suggestion, @s11i-code - I'll probably work that in sometime this week.

In the meantime, if you need an immediate workaround, you can always access the intermediate summaries -- see this passage from the documentation.

If you wish to access the intermediate summaries, you can reference the 'chunks' dictionary included in the response of Chunker.summarize().

from chunkgpt.chunkgpt import Chunker

chunker = Chunker()

text = "..."

summary = chunker.summarize(text)

for chunk in summary['chunks']:
    print(f"CHUNK {chunk}:")
    print("Original Text:")
    print(summary['chunks']['input'])
    print("Summary:")
    print(summary['chunks']['output'])

Hope this helps, and thanks again for the suggested improvement!

[s11i-code]

Thank you, that's very helpful. I did see the chunks in the output but assumed they were pre-summary. Should read the docs better, especially since the documentation is 💯

[dimays]

Hi @s11i-code , I just wanted to let you know that I've published a new release that includes a feature that should hopefully address your needs!

You can install this new release with a simple

pip install --upgrade chunkgpt

The documentation has been updated with more information on this change.

If you end up adopting this new solution, please let me know how it goes!

[s11i-code]

Thank you so much! Looks good. I will use it the next time I need summarization and report back.