AnonymerNiklasistanonym / Md2Anki

Convert Markdown formatted documents to anki decks
https://pypi.org/project/md2anki/
MIT License
6 stars 1 forks source link

The `<` symbol raises an exception #24

Closed baldachin closed 1 year ago

baldachin commented 1 year ago

Exception during document conversion User warning: Field contains the following invalid HTML tags. If your field data has not been HTML encoded, make sure you call html.escape(): <t<1s\)when< p=""> </t<1s\)when<> warnings.warn("Field contained the following invalid HTML tags. Make sure you call html.escape() if" It should be < in the content that throws the exception.

image

AnonymerNiklasistanonym commented 1 year ago

Can you maybe provide the input text (or a simple example) that threw the error? I just tried it with the latest wip version in a normal text and it looks like the problem was maybe fixed but just never pushed.

https://github.com/AnonymerNiklasistanonym/Md2Anki/blob/1e3510580e61cea7a85d83efc957ff1f759c970d/examples/math_formulas_example.md?plain=1#L27-L30

image

baldachin commented 1 year ago

Sorry I didn't find the original documentation on this issue and wasn't able to reproduce it. However, during testing another problem occurred described below:

An error is returned after executing the command:

RuntimeError: Not all code sections were inserted back! (code_sections={2: '<div class="highlight highlight_inline" style=""><pre style="line-height: 125%;"><span></span>JMP FAR PTR OPR </pre></div>', 3: '<div class="highlight highlight_inline" style=""><pre style="line-height: 125%;"><span></span>JMP DWORD PTR[4000H] </pre></div>'})

Cue unusual title content:

## 03-17)如果条件跳转指令的跳转目标范围超出 -128~127 的范围,该如何处埋?举例说明。

-----

段内相对短转移和条件转移的转移范围限制在 -128~127字节,超出该范围就可以改为其他转移方式。

- 段内相对近转移
  `JMP NEAR PTR OPR ; OPR 是段内某个标号` 

- 段内间接转移
  `JMP WORD PRT OPR ; 例如 JMP WORD PTR[SI]` 

- 段间转移

    - 段间直接转移,例如 `JMP FAR PTR OPR` 

    - 段间间接转移,例如 `JMP DWORD PTR[4000H]`  
baldachin commented 1 year ago

I found the file with the anomaly. 04730_example_01.md

AnonymerNiklasistanonym commented 1 year ago

I found the file with the anomaly. 04730_example_01.md

Hmm I kinda got no error at all running that file with the wip version of the program when I just added it as a custom_example.md file into the examples directory and ran all examples.

AnonymerNiklasistanonym commented 1 year ago

Sorry I didn't find the original documentation on this issue and wasn't able to reproduce it. However, during testing another problem occurred described below:

An error is returned after executing the command:

RuntimeError: Not all code sections were inserted back! (code_sections={2: '<div class="highlight highlight_inline" style=""><pre style="line-height: 125%;"><span></span>JMP FAR PTR OPR \</pre\></div>', 3: '<div class="highlight highlight_inline" style=""><pre style="line-height: 125%;"><span></span>JMP DWORD PTR[4000H] \</pre\></div>'})

Cue unusual title content:

## 03-17)如果条件跳转指令的跳转目标范围超出 -128~127 的范围,该如何处埋?举例说明。

-----

段内相对短转移和条件转移的转移范围限制在 -128~127字节,超出该范围就可以改为其他转移方式。

- 段内相对近转移
  `JMP NEAR PTR OPR ; OPR 是段内某个标号` 

- 段内间接转移
  `JMP WORD PRT OPR ; 例如 JMP WORD PTR[SI]` 

- 段间转移

    - 段间直接转移,例如 `JMP FAR PTR OPR` 

    - 段间间接转移,例如 `JMP DWORD PTR[4000H]`  

I'm gonna check this later, getting the same error but not sure why

baldachin commented 1 year ago

I found the file with the anomaly. 04730_example_01.md

Hmm I kinda got no error at all running that file with the wip version of the program when I just added it as a custom_example.md file into the examples directory and ran all examples.

What is 'wip version'? How can I try this version?

AnonymerNiklasistanonym commented 1 year ago

All my heavily untested changes between major releases can be found in this repository on the wip git branch.

With the following commands you should be able to test/use it:

git switch wip

# Linux
python -m venv venv_build
source venv_build/bin/activate
# Windows
python -m venv venv_build
./venv_build/Scripts/Activate.ps1

python -m pip install --upgrade build
python -m build

# Uninstall old version
python -m pip uninstall md2anki
# Install local version
python -m pip install dist/md2anki-$INSERT_CURRENT_VERSION-py3-none-any.whl
AnonymerNiklasistanonym commented 1 year ago

I found the error which was that somehow the markdown to html conversion provided by the Markdown python package converted parts of the placeholders begin/end (<div>.../...</div>) thing to something else (&lt;...). It looks like I can just remove that from the placeholder being/end part though so maybe that just fixes it but I need to run some more tests to see if I suddenly broke anything I'm currently not aware of.

AnonymerNiklasistanonym commented 1 year ago

I just pushed the changes to main for a new pypi release in case you want to check if this fixes your problems.

baldachin commented 1 year ago

I have installed version 3.0.14b0 as per the above guidelines and am testing the question bank files at hand one by one. There is another exception in one of the files:

UserWarning: Field contained the following invalid HTML tags. Make sure you are calling html.escape() if your field data isn't already HTML-encoded: <t<1s\) p="" 时<=""> </t<1s\)> warnings.warn("Field contained the following invalid HTML tags. Make sure you are calling html.escape() if"

which is placed in the attachment. 04730_example_01.md

AnonymerNiklasistanonym commented 1 year ago

It looks like the math section content is not html escaped so the solution would be to just import html, html.escape(math_content). I'm gonna check with some of my old notes if this results in any obvious errors otherwise I can push it. If you wanna try it out check out the git branch wip-escape-tags: https://github.com/AnonymerNiklasistanonym/Md2Anki/tree/wip-escape-tags

https://github.com/AnonymerNiklasistanonym/Md2Anki/blob/17b5fb169dcbb44806dcc710c2755f7d365555b4/src/md2anki/preprocessor.py#L267-L269

AnonymerNiklasistanonym commented 1 year ago

Everything looks good on first sight but I found another bug so the release of the bugfix will be delayed until I fix that one too.

AnonymerNiklasistanonym commented 1 year ago

With the new version release on pypi this problem should be fixed now.

baldachin commented 1 year ago

The file I have on hand has been tested without an anomaly.