littleK0i / SnowDDL

Declarative database change management tool for Snowflake
Apache License 2.0
102 stars 24 forks source link

Error parsing sample files on a windows machine #18

Closed seediang closed 2 years ago

seediang commented 2 years ago

Describe the bug There is an issue parsing the example files under sample01_01 when using Python 3.10 on a windows 11 machine.

The command line used was snowddl -c sample01_01 -a *** -u -p **# --apply-unsafe plan

and the issue is in snowddl_db\bookings\table\flights.yaml and appears to be related to the comment block and the quote characters

as a “connecting flight”: if the

Expected behavior I am expecting SnowDDL to work on a windows 11 machine.

Attach log snowddl -c sample01_01 -a ******* -u **** -p ******# --apply-unsafe plan 2022-08-13 15:11:01.108 - WARNING - [littleK0i\SnowDDL\snowddl\_config\sample01_01\snowddl_db\bookings\table\flights.yaml]: Traceback (most recent call last): File "littlek0i\snowddl\snowddl\parser\abc_parser.py", line 37, in parse_schema_object_files file = ParsedFile(self, path, json_schema) File "littlek0i\snowddl\snowddl\parser\_parsed_file.py", line 26, in __init__ self._load_params() File "littlek0i\snowddl\snowddl\parser\_parsed_file.py", line 45, in _load_params self.params = safe_load(f) or {} File "littleK0i\SnowDDL\.venv\lib\site-packages\yaml\__init__.py", line 125, in safe_load return load(stream, SafeLoader) File "littleK0i\SnowDDL\.venv\lib\site-packages\yaml\__init__.py", line 79, in load loader = Loader(stream) File "littleK0i\SnowDDL\.venv\lib\site-packages\yaml\loader.py", line 34, in __init__ Reader.__init__(self, stream) File "littleK0i\SnowDDL\.venv\lib\site-packages\yaml\reader.py", line 85, in __init__ self.determine_encoding() File "littleK0i\SnowDDL\.venv\lib\site-packages\yaml\reader.py", line 124, in determine_encoding self.update_raw() File "littleK0i\SnowDDL\.venv\lib\site-packages\yaml\reader.py", line 178, in update_raw data = self.stream.read(size) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1776.0_x64__qbz5n2kfra8p0\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 1713: character maps to <undefined>

Attach YAML config (if applicable) I have used the sample01_01 config files.

Notes From the error message looks like an encoding failure, my guess is python on windows is attempting to open the file as CP1252 and not utf8. Adding an explicit encoding='utf8' in the open command at _parsed_file.py, line 44 got it working for me.

littleK0i commented 2 years ago

Fixed in 0.9.1. Thank you for finding this.

I'll consider adding Windows to GitHub actions automated tests.

seediang commented 2 years ago

Thanks for the quick response. The new version is all working on my windows machine.