Measurement problem when determining wrapping position

[lokedhs]

The wrapping mechanism does not take the entire string into account when measuring the width. This causes problems when mechanisms such a glyph composition or kerning is applied.

The problem happens when the sum of the width of each character is not equal to the width of the entire string. The following screenshot illustrates the problem:

Here, the ligature "fi" takes less horizontal space than the inidividual characters f and i. This results in the wrapping happening too early.

Adding a U+200C ZERO WIDTH NON-JOINER between the characters prevents the ligature, rendering the two characters individually, and we can see that the location of the wrap happens at the expected column.

The root cause of this problem could potentially cause problems when the full Unicode word breaking algorithm is introduced, since it also depends on context to a greater degree than the current implementation.

Thoughts on solution

One potential solution to this issue would be to no perform any wrapping computation at all and simply collect characters until some property of the stream changes (explicit flush, change of font, insertion of graphics, etc) and then perform wrapping. That would ensure that all measurement and word-wrapping take place when all the text is available.

[lokedhs]

Here is the source code for the example above:

(defpackage :clim-test.wrap
  (:use :cl)
  (:export #:open-foo-frame))

(in-package :clim-test.wrap)

(declaim (optimize (speed 0) (safety 3) (debug 3)))

(eval-when (:compile-toplevel :load-toplevel :execute)
  (unless (find-package "CLIM")
    #+sbcl
    (progn
      (sb-ext:restrict-compiler-policy 'safety 3)
      (sb-ext:restrict-compiler-policy 'debug 3))
    (ql:quickload "mcclim"))
  (unless (find-package "LOG4CL")
    (ql:quickload "log4cl")))

(eval-when (:compile-toplevel :load-toplevel :execute)
  (defclass foo ()
    ()))

(defun display-text-content (frame stream)
  (declare (ignore frame))
  (loop
    repeat 100
    do (format stream "~c~c" #\f #\i))
  (format stream "~%")
  (loop
    repeat 100
    do (format stream "~c~c~c" #\f #\ZERO_WIDTH_NON-JOINER #\i)))

(clim:define-application-frame foo-frame ()
  ()
  (:panes (text-content :application
                        :display-function 'display-text-content
                        :end-of-line-action :wrap
                        :end-of-page-action :allow))
  (:layouts (default (clim:vertically ()
                       text-content))))

(defun open-foo-frame ()
  (let ((frame (clim:make-application-frame 'foo-frame
                                            :width 600
                                            :height 200)))
    (clim:run-frame-top-level frame)))

[lokedhs]

The problem is hard to see when using the TTF renderer since only the freetype renderer support shaping. However, changing the characters from "fi" to "Av" (upper case A followed by lower case v) introduces kerning, which exposes the same problem using the TTF renderer.