Control characters trigger spurious EOF

TeamSpen210 / srctools

Modules for working with Valve's Source Engine file formats.

Other

55 stars 16 forks source link

Control characters trigger spurious EOF #29

Closed UnBeatWaterGH closed 4 weeks ago

UnBeatWaterGH commented 3 months ago

This is on the latest pip version. If I try to parse TF2's tf_english.txt file I get a weird error:

srctools.keyvalues.KeyValError: End of text reached with remaining open sections.

File ended with at least one keyvalue that didn't have an ending "}".
Open properties: 
- Root at line 1
- "lang" on line 2
- "Tokens" on line 5

TeamSpen210 commented 3 months ago

That is supposed to be keyvalues, yes, but I've had issues in the past with Valve just using invalid syntax in localisation files. Okay, looks like this starts using a bunch of control characters at about line 500, must be confusing my code.

TeamSpen210 commented 3 months ago

As a workaround, you could do sys.modules['srctools._tokenizer'] = None before importing to disable the Cython-accelerated parser. The pure-Python version doesn't have the issue, but is much slower.