This commit adds a check for large files (> 64kb) to check that they are valid inputs. Previously, Trieste would try to read an input file into main memory as a string, regardless of whether that file actually contains something the parser will consider valid. With this change, Trieste will first try parsing the first 4kb of any file over 64kb to see if the parser largely accepts it, and only then continue parsing the rest of the file.
This commit adds a check for large files (> 64kb) to check that they are valid inputs. Previously, Trieste would try to read an input file into main memory as a string, regardless of whether that file actually contains something the parser will consider valid. With this change, Trieste will first try parsing the first 4kb of any file over 64kb to see if the parser largely accepts it, and only then continue parsing the rest of the file.