vlty: integrate Vim spellfile and Yalafi to avoid annoing MORFOLOGIK_RULE

[petRUShka]

Is your feature request related to a problem? Please describe it. Suppose I have to use some science or other slang that is beyond standard spellfiles. I add slang words to Vim spellfile via zg so I do not see these words highlighted in my Vim instance. But in case of using vlty I see Rule ID: MORFOLOGIK_RULE_RU_RU or Rule ID: MORFOLOGIK_RULE_EN_US.

Describe the solution you'd like The idea is to parse spellfile and send it content to LanguageTool spell files before starting grammar and spell check.

I think it is possible with help of these docs.

Describe alternatives you've considered Alternative which is more like a dirty workaround is to turn off these rules (I think with --disablecategories YaLafi command line option).

Additional context Minimal tex:

\documentclass{article}
\begin{document}

Sorry for my bed English. Groebner basis.

\end{document}

Add Groebner to Vim spellfile via zg and then run vlty.

P.S. Possible @matze-dd will be interested in this issue.

[matze-dd]

In principle, this should be feasible; however, it probably requires some substantial work. (Side note: For the YaLafi predecessor Tex2txt, I had a large integration Bash script that did a similar thing as you propose. It is described here, option --adapt-lt. With this option, some private spellfile is appended to LanguageTool's one.)

In the meanwhile: did you see the proposal here for vimtex? YaLafi supports a phrase replacement file, and you could replace/remove your technical terms there. I understand, however, that this causes double work.

I think that this is probably more a proposal/issue for YaLafi, as integration with LanguageTool is done there. (On the other hand, spellfile parsing in Vim script would also suit here. But it could also be done in Python ;-)

EDIT. A reason for implementing all in the vimtex compiler: the code could be reused for interfaces to TeXtidote and others possibly to come.

[petRUShka]

I agree more on vimtex side because it's more vim issue (to extract spellfile content).

In the meanwhile: did you see the proposal here for vimtex? YaLafi supports a phrase replacement file, and you could replace / remove your technical terms there. I understand, however, that this causes double work.

That is very very strange way to do it. I doubt that could be efficient on distance.

[lervag]

The idea is to parse spellfile and send it content to LanguageTool spell files before starting grammar and spell check.

I agree more on vimtex side because it's more vim issue (to extract spellfile content).

That is very very strange way to do it. I doubt that could be efficient on distance.

I have to admit I don't really understand how to approach this feature request. If we did this on the vimtex side, what would we need to do?

If this is more clear for @matze-dd or anyone else I would be very happy to get help with this.

[matze-dd]

This indeed might be a very comfortable feature. I had a look at the docs and tried a simple example. A first toy version could be as follows.

Step 1: Parse the private spellfile.

Assume we have the Russian text (cf. Galerkin orthogonality)

Галёркин

This is marked both by Vim's spell checker and LanguageTool. After saying zg for Galerkin, the file .vim/spell/ru.utf-8.add contains

Галёркин

The file name part utf-8 depends on Vim's encoding, not on the file encoding. After forbidding Galerkin with zw, the file is

#<93>алёркин
Галёркин/!

More syntax elements: :help spell-wordlist-format.

Step 2: Write the list of allowed words to .../LanguageTool-x.x/org/languagetool/resource/ru/hunspell/spelling_custom.txt.

The files *_custom.txt are supported since LT version 4.8. Now, LT also does not complain anymore.

For real life, additional work has to be done. Some first points:

• Fiddling with the location and names of the private spellfiles.
• Fiddling with the path of the local LT installation.
• Writing the 'spelling_custom.txt' does not work, if LT has been installed by pacman. It is not writable. --> sudo?
• If other applications / people use LT, then just overwriting the files might be inappropriate.
• What about the other files ignore.txt and prohibited.txt in the doc?

Altogether, quite some work, if one wants to get it robust!

EDIT. Is there a hook in Vim, so that one could do all the work, if the user says :make? Then, all spellfile modifications would be captured. This could also be useful to postpone the assembly of makeprg in vlty.vim to that point in time.

2nd EDIT. TeXtidote provides an option --dict with a file containing words to be accepted. Therefore, one could support vimtex's textidote compiler in a very similar way.

[matze-dd]

That is very very strange way to do it.

Well, it depends. Since the German LT is fortunately very sensitive, it can, for instance, check gender and case of a noun and the preceding adjective. Replacing technical terms with 'normal-life' words in correct forms does help LT to detect grammar mistakes with technical terms. This works very well in my larger projects.

[petRUShka]

Looks very promising!

Couldn't we use this option of languagetool:

 'lang-xx-dictPath' - absolute path to the hunspell .dic file, use language code instead of 'xx', e.g.
                                lang-tr-dictPath=/path/to/tr.dic. Note that the same directory also needs to
                                contain a common_words.txt file with the most common 10,000 words (used for better language detection)

to use different LT spellfile? Which we can create from original one and vim one combined?

P.S. One more option: hook that force zg not only add word to vim spellfile but also to LT spellfile. Or just to have command to add word to LT spellfile.

[matze-dd]

Couldn't we use this option of languagetool ... One more option: hook that force zg ... Or just to have command to add word to LT spellfile

I agree. These are all options to be considered. A lot of work ! :)

[matze-dd]

3rd EDIT to the post above: For Step 1, I did look for Vim script functions that do the work. Could not find anything, but there is a command :spelldump that unfortunately has no parameter to only dump the private entries. Question in summary: Possibly, there is an internal Vim interface for the spellfiles?

[lervag]

I'm sorry to say I don't really have much knowledge on the spell internals. I still don't quite understand/know if I should read any specific tasks for myself in this issue. I'll be happy to help/pitch in, but it may be useful with specific questions/requests.

@matze-dd I've recognized you post some questions, but I'm sorry to say I don't know any of the answers.

[matze-dd]

As indicated above, implementation of the feature would require some work. Perhaps, the current state is sufficient.

[lervag]

I'll leave it to @petRUShka to answer that :)

[petRUShka]

I agree with both:

• it would require some work
• current state (unnecessary spellchecking lines in output) isn't sufficient enough :)

[matze-dd]

OK, no problem.

[petRUShka]

I probably was misunderstood :) Sorry.

My idea was to mention that current state isn't ok. So IMHO issue should be opened.

[lervag]

@petRUShka Then why did you say "I agree with both". We misunderstand you because you are not being clear.

[lervag]

Also, it is still unclear to me how to actually address this issue. I'm not sure this is something I will prioritize to work on.

[petRUShka]

Then why did you say "I agree with both". We misunderstand you because you are not being clear.

That is why I wrote "sorry" for my unclearness :(

[lervag]

No worries. There are quite a few comments in this thread, and as mentioned, I am not sure how to process it. We'll see what @matze-dd says.

[matze-dd]

We'll see what @matze-dd says.

During the discussion above, we have named some possibilities. In case someone starts implementation, it might be a good idea to choose a general (file-based?) interface working for "plain" LanguageTool, TeXtidote, YaLafi, and others possibly to come.

Currently, I'm not interested in further integration work.

[lervag]

Thanks for your input, @matze-dd.

@petRUShka: I think perhaps one strategy for increasing the likelihood of this issue being resolved is to open a new issue that is a condensed summary of the important "conclusions" from this thread. This would increase the chance that I choose to look into it, at the very least.