oss-slu / esp

MIT License
0 stars 9 forks source link

Implement end pattern detection to improve section extraction accuracy #89

Closed SiriChandanaGarimella closed 1 month ago

SiriChandanaGarimella commented 1 month ago

Is your feature request related to a problem? Please describe. When extracting sections from the log files, the current process sometimes captures irrelevant information. It continues reading lines even after the relevant section has ended, which can lead to including irrelevant data in our extracted content. This makes it difficult to get precise, targeted information from our logs.

Describe the solution you'd like We need to enhance our extract_sections function to include a mechanism for detecting the end of a relevant section.

  1. Add a new function end_pattern() to the extract_sections function.
  2. Modify the content extraction logic to stop when the end_pattern is encountered.