Our usage of bufio.Scanner relies on the default MaxScanTokenSize, which is 64 KiB and may be exceeded, for instance if we encounter a package with a large enough extended description. As this issue may not be prevalent enough to necessitate surfacing extra configuration knobs to the user, I am proposing the use of a reasonable default value (1 MiB) for the time being.
Resolves #49
Our usage of
bufio.Scannerrelies on the defaultMaxScanTokenSize, which is 64 KiB and may be exceeded, for instance if we encounter a package with a large enough extended description. As this issue may not be prevalent enough to necessitate surfacing extra configuration knobs to the user, I am proposing the use of a reasonable default value (1 MiB) for the time being.