Closed JJenkx closed 6 months ago
Thank you for your feedback and for the awk script. I will take a look at this asap in the next day(s). Looks like an issue that I have to fix. The CRLF pair can be tricky to work with sometimes in text processing code for various reasons. I suppose a CR ends the line and cuts the output of the next line for some reason. It's a bit of a surprise, since I have many tests to check ugrep, beyond the ones included with the installation scripts.
If this issue is somehow related to this specifically formatted data, you can use this script to get more similar data. https://github.com/JJenkx/scripts/blob/main/subs
Send var "$formatted_transcript_file" to output file in function "read_and_format_transcript_file"
I found the problem. The problem is caused by an input buffer shift to consume large files, which I've tested in the past. But in this case there is a hiccup in the logic to produce the "before lines context", something like an "off by one error". That will be fixed and released with version 5.0 which I've worked on daily for the last weeks to increase performance, consuming most of my time for testing and tuning (rinse and repeat).
Problem fixed. The patch is to change two lines in ugrep.cpp in ContextGrepHandler::operator()
to adjust the logic as follows:
// functor invoked by the reflex::AbstractMatcher when the buffer contents are shifted out, also called explicitly in grep::search
virtual void operator()(reflex::AbstractMatcher& matcher, const char *buf, size_t len, size_t num) override
[...]
// if we only need the before context, then look for it right before the current lineno
if (state.after_length >= flag_after_context)
{
size_t current = matcher.lineno();
if (lineno + flag_before_context + 1 < current) // FIX
lineno = current - flag_before_context - 1; // FIX
}
The fix will be included in 5.0.
Edit: After changing file from LF to CRLF the ug output properly outputs before line
Line 8803 should contain text. When I did this same test on a file with just a few lines using the same data the desired before line returned properly. See attached file used in my test case awk.output.txt