search

Sigil-Ebook / Sigil

Sigil is a multi-platform EPUB ebook editor
GNU General Public License v3.0
5.99k stars 578 forks source link

[Bug]: Soft hyphens showing up in headers #748

Closed AJDol closed 7 months ago

AJDol commented 7 months ago

Bug Description

Screenshot_20240403_145956, Soft hyphens in headers of my cookbook are showing in Sigil Preview window and Kobo e-readers. This happens only in headers, not in other text on the page. The used font is: Almaq Rough. Soft hyphens are inserted with the Calibre Polishing function in the entire text, not only the headers.

Platform (OS)

Windows (Default)

OS Version / Specifics

Windows 10 Pro 22H2 19045.4170

What version of Sigil are you using?

Since 2.1.0, 2024.04.01 15:40:18 UTC on Qt 6.6.2

Any backtraces or crash reports

none
BeckyDTP commented 7 months ago

This is font issue, not Sigil.

image

AJDol commented 7 months ago

Hallo Becky ,

I respectfully disagree. Why are the soft-hyphens in the last word not showing?

The behavior in the Sigil viewer is erratic, sometimes showing hyphens, sometimes not.

When I remove one soft-hyphen, another is showing up in the same word.

Next time a reload a page a new soft-hyphen is visible.

There must by an issue in the new version of Sigil, because I’ve never noticed this before with the same font.

Best regards,

Antoni

From: Becky @.> Sent: donderdag 4 april 2024 11:42 To: Sigil-Ebook/Sigil @.> Cc: AJDol @.>; Author @.> Subject: Re: [Sigil-Ebook/Sigil] [Bug]: Soft hyphens showing up in headers (Issue #748)

This is font issue, not Sigil.

image.png (view on web) https://github.com/Sigil-Ebook/Sigil/assets/22416021/727e67a1-54f1-477a-a196-76a0dd768e60

— Reply to this email directly, view it on GitHub https://github.com/Sigil-Ebook/Sigil/issues/748#issuecomment-2036695626 , or unsubscribe https://github.com/notifications/unsubscribe-auth/BHSQQEAPDJGPFZC3KJL2QXTY3UN6DAVCNFSM6AAAAABFW35FDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZWGY4TKNRSGY . You are receiving this because you authored the thread. https://github.com/notifications/beacon/BHSQQEDKR7J524A22FZZ5S3Y3UN6DA5CNFSM6AAAAABFW35FDWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTZMWBEU.gif Message ID: @. @.> >

BeckyDTP commented 7 months ago

Why are the soft-hyphens in the last word not showing? I might suspect that this is the whole idea of the problem. A certain randomness in the display of just that particular font. When I remove the first shy-hyphen, I get this view: image

or this: image

I work in Sigil every day, and if such a problem occurred in other fonts, I would certainly notice it. But of course it could be something unique to Qt6.

dougmassay commented 7 months ago

You also mentioned Kobo was doing the same thing. It's unlikely that Sigil AND Kobo broke at the same time.

It's simple enough to test.

Try the same code with no embedded font and try the same code with a different embedded font (one with Unicode support for the soft hyphen character).

AJDol commented 7 months ago

Hello Backy,

These double hyphens are interesting. I don’t think Calibre would enter double hyphens in a header. May Sigil now does under certain conditions?

For me, this is also the first time I encounter this problem, and I’d never seen it before I installed this last version.

So, I cannot be the font. I must be Sigil. A change in the hyphenation routine maybe? A new dictionary?

I would be happy to adjust my process if I know wat Sigil is doing.

Please escalate this issue to someone who knows what is changed in Sigil 2.1.0 or perhaps Qt 6.6.2? My prior install was Sigil-2.0.2.

Thanks in advance,

Antoni

From: Becky @.> Sent: donderdag 4 april 2024 12:24 To: Sigil-Ebook/Sigil @.> Cc: AJDol @.>; Author @.> Subject: Re: [Sigil-Ebook/Sigil] [Bug]: Soft hyphens showing up in headers (Issue #748)

Why are the soft-hyphens in the last word not showing? I might suspect that this is the whole idea of the problem. A certain randomness in the display of just that particular font. When I remove the first shy-hyphen, I get this view: image.png (view on web) https://github.com/Sigil-Ebook/Sigil/assets/22416021/0803be15-1d2a-42ac-bff8-3ddb10013d8a

or this: image.png (view on web) https://github.com/Sigil-Ebook/Sigil/assets/22416021/fd2179ae-5d69-49fe-ab1f-6dc2073610bb

I work in Sigil every day, and if such a problem occurred in other fonts, I would certainly notice it. But of course it could be something unique to Qt6.

— Reply to this email directly, view it on GitHub https://github.com/Sigil-Ebook/Sigil/issues/748#issuecomment-2036784618 , or unsubscribe https://github.com/notifications/unsubscribe-auth/BHSQQEFME4P32R53WHMJQSDY3US3DAVCNFSM6AAAAABFW35FDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZWG44DINRRHA . You are receiving this because you authored the thread. https://github.com/notifications/beacon/BHSQQEHPUB3F4Q3W5PCCKF3Y3US3DA5CNFSM6AAAAABFW35FDWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTZM3O6U.gif Message ID: @. @.> >

BeckyDTP commented 7 months ago

It could be something with Qt6 though, because I checked on 1.9.1 (Qt 5.12.9) and everything is OK there.

image

BeckyDTP commented 7 months ago

2.0.2 (issue exist): image

dougmassay commented 7 months ago

I'm still not getting the Kobo ereader connection.

Is the font being embedded, or just used in Sigil's Preview preferences?

AJDol commented 7 months ago

Unfortunately, I can’t test on a Kobo device myself, but my client mentioned that she saw those soft-hyphens there:

“Only in Kobo did I see strange dashes in the chapter titles. I will send a screenshot. No idea how that is possible and why it is not the case with the other viewers.”

The screenshot is from her phone using the Kobo app. The font is embedded in the e-book.

It's in Sigil's preview in any case.

Best regards, Antoni

From: Doug Massay @.> Sent: donderdag 4 april 2024 13:23 To: Sigil-Ebook/Sigil @.> Cc: AJDol @.>; Author @.> Subject: Re: [Sigil-Ebook/Sigil] [Bug]: Soft hyphens showing up in headers (Issue #748)

I'm still not getting the Kobo ereader connection.

Is the font being embedded, or just used in Sigil's Preview preferences?

— Reply to this email directly, view it on GitHub https://github.com/Sigil-Ebook/Sigil/issues/748#issuecomment-2036901235 , or unsubscribe https://github.com/notifications/unsubscribe-auth/BHSQQEA4NFPHSZDFU7E5IVLY3UZ23AVCNFSM6AAAAABFW35FDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZWHEYDCMRTGU . You are receiving this because you authored the thread. https://github.com/notifications/beacon/BHSQQEBYP4P6OPQ7GTS5FOTY3UZ23A5CNFSM6AAAAABFW35FDWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTZNCSXG.gif Message ID: @. @.> >

BeckyDTP commented 7 months ago

Screenshot_2024-04-04-14-26-37-869_com kobobooks android

Screenshot from kobo app on Android. Issue exist.

kevinhendricks commented 7 months ago

If it is an embedded font and the soft-hyphens only show up in headers, it is because the header causes a font size that is most likely synthesized by the font engine used by Sigil or the e-reader. In this case the Windows DirectWrite font engine.

BeckyEbook, would you try setting the environment variable to use the freetype font engine instead of the Windows one to see if that has any impact?

Then changing the header sizes to see if that impacts anything as some font sizes might not need to be synthesized.

Removing soft-hyphens from headers text should be an easy find and replace.

kevinhendricks commented 7 months ago

Is there a test case epub attachment here with that font included. The original font link is loaded with malware and crap links.

dougmassay commented 7 months ago

Also note that the version of Almaq Rough on that website is an evaluation copy only. It's not freely redistributable in commercial endeavors without paying for a license.

AJDol commented 7 months ago

Here it is. (I've attached it to the original post at the top of the screen). For now, I have removed soft hyphens from headers.

From: Kevin Hendricks @.> Sent: donderdag 4 april 2024 14:34 To: Sigil-Ebook/Sigil @.> Cc: AJDol @.>; Author @.> Subject: Re: [Sigil-Ebook/Sigil] [Bug]: Soft hyphens showing up in headers (Issue #748)

Is there a test case epub attachment here with that font included. The original font link is loaded with malware and crap links. — Reply to this email directly, view it on GitHub https://github.com/Sigil-Ebook/Sigil/issues/748#issuecomment-2037069640 , or unsubscribe https://github.com/notifications/unsubscribe-auth/BHSQQEFNGLR6XMDQACJZMQDY3VCD5AVCNFSM6AAAAABFW35FDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZXGA3DSNRUGA . You are receiving this because you authored the thread. https://github.com/notifications/beacon/BHSQQEH6K2TRMBU4EROB7XLY3VCD5A5CNFSM6AAAAABFW35FDWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTZNM3UQ.gif Message ID: @. @.> >

AJDol commented 7 months ago

Check. I got a different version from my client, but wanted to test with a different font to check if it had the same issue. It had.

Antoni

From: Doug Massay @.> Sent: donderdag 4 april 2024 14:59 To: Sigil-Ebook/Sigil @.> Cc: AJDol @.>; Author @.> Subject: Re: [Sigil-Ebook/Sigil] [Bug]: Soft hyphens showing up in headers (Issue #748)

Also note that the version of Almaq Rough on that website is an evaluation copy only. It's not freely redistributable in commercial endeavors without paying for a license.

— Reply to this email directly, view it on GitHub https://github.com/Sigil-Ebook/Sigil/issues/748#issuecomment-2037147858 , or unsubscribe https://github.com/notifications/unsubscribe-auth/BHSQQED6IKL4DFWL44A7O7TY3VFBBAVCNFSM6AAAAABFW35FDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZXGE2DOOBVHA . You are receiving this because you authored the thread. https://github.com/notifications/beacon/BHSQQEFJYXQK3XJHJNV5IALY3VFBBA5CNFSM6AAAAABFW35FDWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTZNRUNE.gif Message ID: @. @.> >

dougmassay commented 7 months ago

Clearly there's something wrong with the font in my opinion. It's possible that whatever is wrong was not exposed in earlier versions of QtWebEngine in Qt5, but IS exposed in Qt6, but ...

We have no idea how long it's been a problem in Kobo's rendering engine. Or other rendering engines for that matter. The Kobo test is proof that its not just a Sigil, or Qt6 glitch.

dougmassay commented 7 months ago

Here it is. For now I have removed soft-hyphens from headers.

I don't think you're going to able to attach an epub via the github email interface. You'll need to visit this thread in the web interface and use the "Attach Files" button.

kevinhendricks commented 7 months ago

If just one of the glyphs for that font for the soft hyphen is filled in, then if that size is closest to the header font size, the synthesized header font will be incorrect? Try bolding and changing font sizes, and you should see if that is the case.

kevinhendricks commented 7 months ago

This may in fact be a chrome bug. Perhaps kobo on Android uses chrome?

dougmassay commented 7 months ago

I believe Kobo has two different rendering engines. One for epubs and one for their proprietary kepub format. One or both could use chrome.

On Thu, Apr 4, 2024, 11:40 AM Kevin Hendricks @.***> wrote:

This may in fact be a chrome bug. Perhaps kobo on Android uses chrome?

— Reply to this email directly, view it on GitHub https://github.com/Sigil-Ebook/Sigil/issues/748#issuecomment-2037552132, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACG3CXQRBJSM4GLKPFUMCRTY3VYATAVCNFSM6AAAAABFW35FDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZXGU2TEMJTGI . You are receiving this because you commented.Message ID: @.***>

kevinhendricks commented 7 months ago

Either way, I grabbed that font from that crappy download site, then loaded it into FontForge and it shows a "hyphen" glyph for the unicode character 173, 0xAD (soft hyphen) in the cmap. That character should not in fact have any glyphs as it is a zero width placeholder for possible hyphenation. Some fonts do include a glyph there but then mark the character as unprintable. As far as I can tell, it is not marked as such.

So whatever the problem, it is related to this font and how it is processed by various font engines.

AJDol commented 7 months ago

Thx for your research. I will have to account for that. For now, all soft hyphens are removed from headers.

kevinhendricks commented 7 months ago

To test things, I used Fontforge to clear the glyph for the soft hyphen and then save it to the font. I then ran the font on a test case with soft hyphens and it now works and hyphenates correctly in Sigil / Chrome.

So I do believe that some font rendering systems, ignore glyphs when they exist in the soft hyphen location while other font renderers use what is there regardless.

I have no idea which is technically more correct.

BTW, FontForge found lots of other errors and warnings when operating with this font such as improperly mapped glyphs/cmaps, unclosed points, etc. I recommend staying far away from this font or at least fix all those errors and warnigns using a font enditing and checking tool like FontForge. What editing it would do to the license for this font is then unknown.

So I am closing this since it appears to be issues with that Font that cause some font renderers problems.