debakarr / kodekloud-downloader

Simple downloaded for https://kodekloud.com/
103 stars 34 forks source link

quiz_markdown update with encoding utf-8 #44

Closed Matkach closed 2 months ago

Matkach commented 2 months ago

When I tried to use the command:

kodekloud dl-quiz -o KodeKloudQuiz --sep

I got the following error:

Fetching Quiz 44 - Lab MCQ IAC and Terraform Basics
Traceback (most recent call last):
  File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Python310\Scripts\kodekloud.exe\__main__.py", line 7, in <module>
  File "C:\Python310\lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "C:\Python310\lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "C:\Python310\lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Python310\lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Python310\lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "C:\Python310\lib\site-packages\kodekloud_downloader\cli.py", line 101, in dl_quiz
    download_quiz(output_dir, sep)
  File "C:\Python310\lib\site-packages\kodekloud_downloader\main.py", line 60, in download_quiz
    Path(output_file).write_text(markdown_text)
  File "C:\Python310\lib\pathlib.py", line 1155, in write_text
    return f.write(data)
  File "C:\Python310\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in position 10314: character maps to <undefined>

The error I was encountering is due to the write_textmethod of the Pathobject. This method writes a string to the file, and it uses the default encoding to do so. In this case, the default encoding seems to be ‘cp1252’, which doesn’t support the unicode character ‘\u200b’ (a zero-width space) present in the markdown_text.

To fix this issue, I explicitly specify the encoding as ‘utf-8’ when writing to the file. However, the write_textmethod doesn’t allow me to specify the encoding directly. Instead, I use the built-in openfunction with the ‘utf-8’ encoding.