Installing from source fails

Installing from source fails with the log below. Adding a - in utf8 in mediawiki_parser\preprocessorParser.py fixes it though. Perhaps this is a Pijnu error?

running build_parsers
... parsing grammar ...
name pattern format GT
name pattern format PIPE
name pattern format EOL
name pattern format TAB
... composing code ...
... writing file ...

['mediawiki_parser', 'C:\\Users\\AB1908\\Documents\\GitHub\\mediawiki-parser', 'C:\\Users\\AB1908\\AppData\\Local\\Programs\\Python\\Python38-32\\python38.zip', 'C:\\Users\\AB1908\\AppData\\Local\\Programs\\Python\\Python38-32\\DLLs', 'C:\\Users\\AB1908\\AppData\\Local\\Programs\\Python\\Python38-32\\lib', 'C:\\Users\\AB1908\\AppData\\Local\\Programs\\Python\\Python38-32', 'C:\\Users\\AB1908\\AppData\\Roaming\\Python\\Python38\\site-packages', 'C:\\Users\\AB1908\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\site-packages']
Traceback (most recent call last):
  File "setup.py", line 56, in <module>
    setup(
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\site-packages\setuptools\__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\distutils\dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "setup.py", line 49, in run
    makeparsers(self.force)
  File "setup.py", line 30, in makeparsers
    makeParser(preprocessorGrammar, outputPath="mediawiki_parser")
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pijnu\generator\generator.py", line 168, in makeParser
    module = importlib.import_module(modulename)
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 779, in exec_module
  File "<frozen importlib._bootstrap_external>", line 916, in get_code
  File "<frozen importlib._bootstrap_external>", line 846, in source_to_code
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "mediawiki_parser\preprocessorParser.py", line 0
SyntaxError: 'utf-8' codec can't decode byte 0xa1 in position 12533: invalid start byte

Correcting the error and re-running leads to the same thing happening, but in wikitextParser.py instead. Corresponding log:

running build_parsers
... parsing grammar ...
name pattern format QUOTE
name pattern format GT
name pattern format LT
name pattern format GT
name pattern format EOL
name pattern format TAB
name pattern format TAB
name pattern format TAB
... composing code ...
... writing file ...

['mediawiki_parser', 'C:\\Users\\AB1908\\Documents\\GitHub\\mediawiki-parser', 'C:\\Users\\AB1908\\AppData\\Local\\Programs\\Python\\Python38-32\\python38.zip', 'C:\\Users\\AB1908\\AppData\\Local\\Programs\\Python\\Python38-32\\DLLs', 'C:\\Users\\AB1908\\AppData\\Local\\Programs\\Python\\Python38-32\\lib', 'C:\\Users\\AB1908\\AppData\\Local\\Programs\\Python\\Python38-32', 'C:\\Users\\AB1908\\AppData\\Roaming\\Python\\Python38\\site-packages', 'C:\\Users\\AB1908\\AppData\\Local\\Programs\\Python\\Python38-32\\lib\\site-packages']
Traceback (most recent call last):
  File "setup.py", line 56, in <module>
    setup(
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\site-packages\setuptools\__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\distutils\dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\distutils\dist.py", line 985, in run_command
    cmd_obj.run()
  File "setup.py", line 49, in run
    makeparsers(self.force)
  File "setup.py", line 36, in makeparsers
    makeParser(mediawikiGrammar, outputPath="mediawiki_parser")
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pijnu\generator\generator.py", line 168, in makeParser
    module = importlib.import_module(modulename)
  File "C:\Users\AB1908\AppData\Local\Programs\Python\Python38-32\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 779, in exec_module
  File "<frozen importlib._bootstrap_external>", line 916, in get_code
  File "<frozen importlib._bootstrap_external>", line 846, in source_to_code
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "mediawiki_parser\wikitextParser.py", line 0
SyntaxError: 'utf-8' codec can't decode byte 0xa1 in position 31447: invalid start byte

Again correcting utf8 to utf-8 manually fixes the problem.

Platform: Windows 10 Python Version: 3.8.1

peter17 / mediawiki-parser

Installing from source fails #8