gpoore / codebraid

Live code in Pandoc Markdown
BSD 3-Clause "New" or "Revised" License
367 stars 13 forks source link

Use of inline html code characters appears to break source file parsing #44

Closed rossbar closed 2 years ago

rossbar commented 3 years ago

Markdown files with html coded characters result in a StopIteration error from codebraid.

Minimal reproducing example

$ echo "I need $20." > foo.md
$ cat foo.md
I need $20.
$ pandoc --from markdown --to html foo.md -o foo.html  # Works as expected
$ cat foo.html
<p>I need $20.</p>
$ codebraid pandoc --from markdown --to html foo.md -o foo.html
Traceback (most recent call last):
  File "/home/ross/.virtualenvs/elegant-scipy/bin/codebraid", line 8, in <module>
    sys.exit(main())
  File "/home/ross/.virtualenvs/elegant-scipy/lib/python3.9/site-packages/codebraid/cmdline.py", line 107, in main
    args.func(args)
  File "/home/ross/.virtualenvs/elegant-scipy/lib/python3.9/site-packages/codebraid/cmdline.py", line 153, in pandoc
    converter.code_braid()
  File "/home/ross/.virtualenvs/elegant-scipy/lib/python3.9/site-packages/codebraid/converters/base.py", line 1198, in code_braid
    self._extract_code_chunks()
  File "/home/ross/.virtualenvs/elegant-scipy/lib/python3.9/site-packages/codebraid/converters/pandoc.py", line 1124, in _extract_code_chunks
    self._load_and_process_initial_ast(source_string=source_string, single_source_name=source_name)
  File "/home/ross/.virtualenvs/elegant-scipy/lib/python3.9/site-packages/codebraid/converters/pandoc.py", line 994, in _load_and_process_initial_ast
    source_name, line, line_number = next(source_name_line_and_number_iter)
StopIteration

It seems this is tangentially related to other issues re: handling escape characters e.g. #36. The discussions there and in #38 nicely explain how the StopIteration exception itself comes about.

Version info

Python: 3.9.2 codebraid: 0.5.0 pandoc: 2.13

gpoore commented 3 years ago

Thanks for reporting this. This makes source sync significantly more complicated, so the type of approach I've been using really can't be adapted. I think I have an idea for a more general algorithm that can handle this, and will try to get that implemented as soon as possible. I'll at least fix things soon so that source sync fails without bringing down the whole program.

gpoore commented 3 years ago

This entire category of errors from syncing code to source line numbers should be permanently eliminated by the last commit. I need to work on a few other issues and will then put a new version on PyPI.