averkij / lingtrain-aligner

Lingtrain Aligner — ML powered library for the accurate texts alignment.
GNU General Public License v3.0
123 stars 9 forks source link

Unintended retention of '%%%%%' term after forced paragraph separation #8

Closed ilpoli closed 1 month ago

ilpoli commented 1 year ago

I am using the "%%%%%." term to force separate lines into individual paragraphs. Here is an example:

Neil%%%%%h5.
We’ll discuss that more in a moment and find out if chatbots really think for themselves. But first I have a question for you, Rob. The first computer program that allowed some kind of plausible conversation between humans and machines was invented in 1966, but what was it called? Was it:

a) ALEXA %%%%%.

b) ELIZA %%%%%.

c) PARRY %%%%%.

While this process successfully separates the items into different paragraphs, it doesn't remove the "%%%%%" term, which is consequently retained in the final document.

image
averkij commented 1 month ago

"%%%%%." marks adds only automatically on the lienbreaks. So you don't need to add them directly. Just add line breaks to your text and check that when you generating the book the appropriate side is selected ("from" or "to").