When I run Rustscan for port scanning, some IPs return a large number of open ports. As a result, the generated raw-open-ports.txt file looks like the following:
However, when I run CleanRustScan() to process the above raw-open-ports.txt file, I noticed that the function throws a "bufio.Scanner: token too long" error when processing some very large line data.
Therefore, I suggest setting the maximum token size in the scanner.Buffer() function's second parameter to support reading files with very large line data. For example, you can use the following code:
scanner := bufio.NewScanner(file)
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024)
for scanner.Scan() {
val := strings.TrimSpace(scanner.Text())
if val == "" {
continue
}
result = append(result, val)
}
Hi,
When I run Rustscan for port scanning, some IPs return a large number of open ports. As a result, the generated raw-open-ports.txt file looks like the following:
However, when I run CleanRustScan() to process the above raw-open-ports.txt file, I noticed that the function throws a
"bufio.Scanner: token too long"
error when processing some very large line data.Therefore, I suggest setting the maximum token size in the
scanner.Buffer()
function's second parameter to support reading files with very large line data. For example, you can use the following code: