search

textlint / textlint-plugin-latex2e

Textlint Plugin LaTeX2ε
MIT License
67 stars 8 forks source link

Working on v2 #463

Open pddg opened 7 months ago

pddg commented 7 months ago

I have made some modifications since I became one of the maintainers, but I have not been able to actively participate in the development over the past few years. I had discovered some problems but had put them off due to the complexity of the current code base.

Several major events have occurred during this time.

This project is a viable solution for using textlint in LaTeX, even in the year 2024, with no alternative. I began to think that it might be appropriate to rebuild the entire project once in order to maintain it on an ongoing basis. This means building v2 from scratch.

Which parser library is better for us?

For v2 I re-evaluated several LaTeX parsers written in TypeScript or JavaScript. Sample TeX source is as follows:

% This is the beginning of the document
\documentclass{article}  % This is a comment

\begin{document}

This is a paragraph that contains inline math: $E=mc^2$.
This is a second line.

% This is the beginning of the nested document
\begin{itemize}
    \item First item
    \item Second item
    \item Third item
\end{itemize}

\begin{tabular}{|c|c|c|}
    \hline
    Column 1 & Column 2 & Column 3 \\
    \hline
    Row 1 & Row 1 & Row 1 \\
    \hline
    Row 2 & Row 2 & Row 2 \\
    \hline
\end{tabular}

\end{document}

latex-utensils v6.2.0

Generated AST of sample tex source is here.

unified-latex v1.6.0

Generated AST of sample tex source is here.

completeBlank and completeComment is hard to maintain. I think it might be worth the transition.

LaTeX.js v0.12.6

(As far as I could tell) this could not be used as a mere parser. It may be possible to define a Generator that generates textlint ASTs from LaTeX ASTs. However, I do not know how to create a Generator. Document says TODO, now. https://latex.js.org/api.html#class-generator

Create original one

Another option would be to make your own, but the reason other parsers produce such output in the first place is because that is the way TeX syntax is written. Therefore, it is difficult to imagine that a home-built parser would be significantly easier for us to use.

Goal of v2

Non Goal

How do we do

As maintainers, we should work on one of the following

Which one do you think is better? @tani @kn1cht

tani commented 7 months ago

Thank you for revisiting our project. I would like to express the gratitude for your deep survey and valuable comments.

I agree with your comment that our project depends on the old library and it should be updated to the latest version. Further, I am happy to hear that I could collaborate with you again. Let's reboot our project and make it better.

In my opinion, I would like to use unified-latex libarary. Even if we use the latex-utensils, we need to momdify the upstream code to make it work with our project. I think that it is time to change the library to unified-latex because it is more feature-rich. (For my personal preference, I tend to use the new library for better experience :wink:.)

kn1cht commented 7 months ago

Hello, I am glad to hear that you are doing well! I also agree with working on v2.

pddg commented 7 months ago

whitespace and parbreak around \item has no location information

This may be resolved by parseMinimal method of unified-latex. https://github.com/pddg/eval-js-latex-parsers/blob/e31b34348473fdecbc1f628c07d49fb9373c3f8f/unified-latex-minimal/output.json#L726-L740

pddg commented 7 months ago

Looking at the reference, markdown-to-ast, it appears that whitespace, etc. caused by List nesting is simply ignored. Each ListItem has no whitespace before or after it, and whitespace or line breaks exist only in the raw value of the List. We might simply remove all elements without position from the AST and then convert them.

pddg commented 7 months ago

I have written a sample that works to some extent, but have found additional problems and reported them to unified-latex. https://github.com/siefkenj/unified-latex/issues/77#issue-2140725952

pddg commented 7 months ago

Because latex combines elements of formatting and text, it is often difficult to express in ASTs prepared for mere text.

I am currently struggling with the description environment. What kind of AST should this be expressed as in textlint?

\begin{description}
  \item[item1] desc
  \item[item2] desc
\end{description}

I have three ideas.

1. Consider headings and their descriptions to be lists.

- item1
  - desc
- item2
  - desc

textlint AST will be as follows:

List
  ListItem
    Paragraph
    List
      ListItem
        Paragraph
  ListItem
    Paragraph
    List
      ListItem
        Paragraph

2. Consider the heading as a Header and the description as a Paragraph.

- ## item1
  desc
- ## item2
  desc

textlint AST will be as follows:

List
  ListItem
    Header
    Paragraph
  ListItem
    Header
    Paragraph

3. Consider headings and their descriptions as an independent Paragraph.

- item1
  desc
- item2
  desc

textlint AST will be as follows:

List
  ListItem
    Paragraph
    Paragraph
  ListItem
    Paragraph
    Paragraph
kn1cht commented 7 months ago

Since Re:VIEW also has definition list, I examined the behavior of textlint-plugin-review. As a result, it output AST similar to the idea 1.

input (test/chunker-test.js)

 : Alpha
    DEC の作っていた RISC CPU。
    浮動小数点数演算が速い。
 : POWER
    IBM とモトローラが共同製作した RISC CPU。
    派生として POWER PC がある。
 : SPARC
    Sun が作っている RISC CPU。
    CPU 数を増やすのが得意。

output: It seems that the Re:VIEW plugin generates ListItem element for each header and list item. This is close to idea 1, but the difference is whether the ListItem hierarchy is flat or nested. Personally, I feel that the flat hierarchy is simpler.

pddg commented 7 months ago

@tani @kn1cht I have created PR for v2. At least simple source code parsing and many of the implemented tests seem to work.