liblouis / liblouisutdml

An open-source library providing complete braille transcription services for xml, html and text documents
http://liblouis.io
GNU General Public License v3.0
24 stars 16 forks source link

In Liblouis 3.30 release 18 Liblouisutdml Math Voluve tests are failing #106

Closed hammera closed 5 months ago

hammera commented 5 months ago

Hi Boys,

Before I dropping a general version number update related PR to the Liblouis 3.30 version number the Liblouisutdml repository affected files (Dockerfile.win32, Dockerfile.win64, .github/workflow/main.iml, .github.workflow/sanitizer.iml), I compiled my Debian system a Liblouis 3.30 deb package and install the updated package version. Make command ran successfull the Liblouisutdml source tree, but when I ran the test directory the make check command, I see following test-suite.log:

================================================
   liblouisutdml 2.13.0: tests/test-suite.log
================================================

# TOTAL: 386
# PASS:  187
# SKIP:  0
# XFAIL: 181
# FAIL:  18
# XPASS: 0
# ERROR: 0

Me suspicious happens the failure because happened a change the virtual dots feature related, this situations always the math_voluwe tests produced failures. Bingo. :-):-) Following the failing tests list:

Looks the verbose tests (only the failures part):

FAIL: test_mathml_woluwe/test_051
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_051.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foosWbq3i/output.txt   2024-06-04 13:15:03.942981180 +0200
@@ -1 +1 @@
-@N"*"(#J")"7@N*0
+@N"*"(#J")"7@NA_R0
FAIL test_mathml_woluwe/test_051.test (exit status: 1)

FAIL: test_mathml_woluwe/test_088
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_088.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foomNK5c2/output.txt   2024-06-04 13:15:04.746985928 +0200
@@ -1 +1 @@
-X*N:
+XA_RN:
FAIL test_mathml_woluwe/test_088.test (exit status: 1)

FAIL: test_mathml_woluwe/test_090
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_090.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooH5928L/output.txt   2024-06-04 13:15:04.794986212 +0200
@@ -1 +1 @@
-#*N:X
+A_LN:X
FAIL test_mathml_woluwe/test_090.test (exit status: 1)

FAIL: test_mathml_woluwe/test_092
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_092.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foo4pXn5P/output.txt   2024-06-04 13:15:04.834986449 +0200
@@ -1 +1 @@
-X_*N:
+XA_CN:
FAIL test_mathml_woluwe/test_092.test (exit status: 1)

FAIL: test_mathml_woluwe/test_094
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_094.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooDLv7PI/output.txt   2024-06-04 13:15:04.882986732 +0200
@@ -1 +1 @@
-#*N:X*M:
+A_LN:XA_RM:
FAIL test_mathml_woluwe/test_094.test (exit status: 1)

FAIL: test_mathml_woluwe/test_095
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_095.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooARDilN/output.txt   2024-06-04 13:15:04.902986850 +0200
@@ -1 +1 @@
-#*N:X*M:
+A_LN:XA_RM:
FAIL test_mathml_woluwe/test_095.test (exit status: 1)

FAIL: test_mathml_woluwe/test_097
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_097.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooO4Zk1d/output.txt   2024-06-04 13:15:04.946987112 +0200
@@ -1 +1 @@
-#*N:X_/M:
+A_LN:X_/M:
FAIL test_mathml_woluwe/test_097.test (exit status: 1)

FAIL: test_mathml_woluwe/test_098
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_098.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooCloX1f/output.txt   2024-06-04 13:15:04.970987254 +0200
@@ -1 +1 @@
-X/N:*M:/P:
+X/N:A_RM:/P:
FAIL test_mathml_woluwe/test_098.test (exit status: 1)

FAIL: test_mathml_woluwe/test_101
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_101.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooxXvvIW/output.txt   2024-06-04 13:15:05.038987654 +0200
@@ -1 +1 @@
-X*1,1 X*2
+XA_R1,1 XA_R2
FAIL test_mathml_woluwe/test_101.test (exit status: 1)

FAIL: test_mathml_woluwe/test_102
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_102.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foonPFWac/output.txt   2024-06-04 13:15:05.062987796 +0200
@@ -1 +1 @@
-X*2/3
+XA_R2/3
FAIL test_mathml_woluwe/test_102.test (exit status: 1)

FAIL: test_mathml_woluwe/test_103
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_103.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooV5G522/output.txt   2024-06-04 13:15:05.082987913 +0200
@@ -1 +1 @@
-X*2/3
+XA_R2/3
FAIL test_mathml_woluwe/test_103.test (exit status: 1)

FAIL: test_mathml_woluwe/test_104
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_104.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foo7DTMm8/output.txt   2024-06-04 13:15:05.106988055 +0200
@@ -1 +1 @@
-X*P:Y/Q:
+XA_RP:Y/Q:
FAIL test_mathml_woluwe/test_104.test (exit status: 1)

FAIL: test_mathml_woluwe/test_105
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_105.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooTNnpYQ/output.txt   2024-06-04 13:15:05.130988197 +0200
@@ -1 +1 @@
-A*N"-#A:X/N"-#A:
+AA_RN"-#A:X/N"-#A:
FAIL test_mathml_woluwe/test_105.test (exit status: 1)

FAIL: test_mathml_woluwe/test_108
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_108.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foo8oHED1/output.txt   2024-06-04 13:15:05.198988598 +0200
@@ -1 +1 @@
-A*X"6Y:/Z:
+AA_RX"6Y:/Z:
FAIL test_mathml_woluwe/test_108.test (exit status: 1)

FAIL: test_mathml_woluwe/test_109
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_109.test/expected.txt 2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooyfDiXo/output.txt   2024-06-04 13:15:05.218988718 +0200
@@ -1 +1 @@
-A*X/2"6Y:"6B
+AA_RX/2"6Y:"6B
FAIL test_mathml_woluwe/test_109.test (exit status: 1)

FAIL: test_mathml_woluwe/test_110
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_110.test/expected.txt 2022-12-12 12:31:51.740096764 +0100
+++ /tmp/foo6saQq2/output.txt   2024-06-04 13:15:05.242988860 +0200
@@ -1 +1,2 @@
-F*I:/J:<X>"7A*I:/J:<X>A*I"6#A:/J"6#A:<X>
+FA_RI:/J:<X>"7AA_RI:/J:<X>AA_RI"6#A:/J"-
+6#A:<X>
FAIL test_mathml_woluwe/test_110.test (exit status: 1)

FAIL: test_mathml_woluwe/test_111
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_111.test/expected.txt 2022-12-12 12:31:51.740096764 +0100
+++ /tmp/foo4VMkAS/output.txt   2024-06-04 13:15:05.266989002 +0200
@@ -1 +1 @@
-"2T\X*I:/M:"6X*J:/M::
+"2T\XA_RI:/M:"6XA_RJ:/M::
FAIL test_mathml_woluwe/test_111.test (exit status: 1)

FAIL: test_mathml_woluwe/test_119
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_119.test/expected.txt 2022-12-12 12:31:51.740096764 +0100
+++ /tmp/fooyp7p39/output.txt   2024-06-04 13:15:05.442990041 +0200
@@ -1 +1 @@
-"2T\%A*P:/N::"6%A*Q:/N:::
+"2T\%AA_RP:/N::"6%AA_RQ:/N:::
FAIL test_mathml_woluwe/test_119.test (exit status: 1)

Until this tests are not adopted the new changed tables with affects the dutch virtual dot feature change with released the Liblouis 3.30 version, impossible to increase Liblouis version number to 3.30 version, except not marks xfail flags the affected 18 testcases. But the xfail markup is a temporary workaround only. :-):-) Now have a dedicated maintainer the voluve tests in LiblouisUTDML? Usual this tests are using I think the lbu_files/wiskunde.sem semantic action file and following tables, based from liblouis.ini the voluve tests main directory:

literarytextTable nl-unicode.dis,../../lbu_files/wiskunde.ctb,braille-patterns.cti
mathtextTable     nl-unicode.dis,../../lbu_files/wiskunde.ctb,braille-patterns.cti
mathexprTable     nl-unicode.dis,../../lbu_files/wiskunde.ctb,braille-patterns.cti
editTable en-us-brf.dis,braille-patterns.cti,../../lbu_files/wiskunde_edit.ctb

Bert, have an ydea the fix related? If I remember right you known the voluve tests hierarchy (sure better known with me). :-):-) Samuel Don't like the mark xfail the failing tests type fixes, and will be releasing future a Liblouis version update to the 3.30 version the Debian Backports repository. :-):-) And, a bonus: Lot of changes happened in Liblouis side the 2023 year with based the Liblouisutdml 2.12.0 version, perhaps will be good if we doing future a fresh Liblouisutdml 3.13.0 release with contains all great new Liblouis related tables and languages (turkish table updates, new 3.30 version landed tables, etc), perhaps in jul or perhaps after the september publicated Liblouis 3.31 release. I welcome helping this tasks related with not coding way, because me not have detailed C code experience, and active using Liblouisutdml. :-):-)

Attila

egli commented 5 months ago

@bertfrees what should we do here? These tests fail almost with every release (as the dutch tables change). Are these tests useful?

bertfrees commented 5 months ago

Are the tests useful? There's no question about that! (Unless they are disabled with xfail.) Is it annoying that the test suite fails with almost every Dutch table update involving virtual dots? Yes. Is it annoying that Liblouisutdml is pretty much unmaintained and that it hinders smooth releasing of Liblouis in Debian? Hell yes!

That said, it should be relatively easy to fix. It's probably just some character that is used in wiskunde.sem (some bracket or underscore or something) that has gotten a different virtual dot. It breaks easily also because of the way the Liblouisutdml code is written. It could be improved a lot.

hammera commented 5 months ago

Bert, what the expected output braille sequence the failing tests the new changed virtual outputs related with perhaps affecting wiskunde.sem? wiskunde.ctb or perhaps wiskunde-edit.ctb table are affected, or only the semantic action file or an included another subtable in Liblouis? For Liblouis commit 9b7b15997c56d0772ccd86294d344b22fa14b32a have following commit entry affects the tables/nl-chardefs.uti:

commit 9b7b15997c56d0772ccd86294d344b22fa14b32a
Author: Bert Frees <bertfrees@gmail.com>
Date:   Wed Apr 24 22:21:55 2024 +0200

    Add more virtual dots to superscripts and subscripts

    Virtual dots are added to the superscript/subscript indicators, as
    well as the characters that are lifted/dropped.

Very interesting, when I examining after this feature merge and master branch of Liblouis UTDML cooperation with have a regression LiblouisUTDDML the test entries or not, when I setted up LOUIS_TABLEPATH environment value my /usr/src/liblouis/tables directory, I not experienced failure testcases, only when I do a full Liblouis 3.30 package. I doed wrong the previous test before official 3.30 Liblouis release comed out? Unfortunatelly, very difficult to doing a rolling development Liblouis package for debian, but perhaps I future will be trying doing my PPA a packaging recipe and for example a liblouis-git renamed package if not have better way to early we detect this issues future. :-(:-( In Arch Linux lot of a11y packages have installable development snapshot packages, for example orca-git. But me don't want using Arch (I used it all the time, when a rolling update killed my entire sound system on a virtual machine, it's a good thing that I didn't use it as a production system). :-):-) Anyway, for the future, I will solve this set of packages with daily updates for my own system, so that we can notice these problems in time, so that these phenomena do not have to be discovered afterwards. If there is no capacity to fix these tests now, should I temporarily put them in xfail state and write a comment in the final commit with a list of tests, referring to this bug ticket, so that we don't forget to fix these tests later?

egli commented 5 months ago

If there is no capacity to fix these tests now, should I temporarily put them in xfail state and write a comment in the final commit with a list of tests, referring to this bug ticket, so that we don't forget to fix these tests later?

@hammera can you look at the test results and see if you can fix them easily? Maybe just add the virtual dots to the expected results?

hammera commented 5 months ago

Ok Chris, I welcome trying investigate and fixing this issue if I get a little technical help, because I am not known detailed yet this tests how depending dutch virtual dots related with this tests, but I welcome learning new techniques. :-):-) I need add proper virtual dots the semantic action file, or the test case entry expected.txt file? Bert, in Liblouis side what subtable affecting this 18 tests? The nl-chardefs.uti file, or I need search the proper virtual dots the braille-patterns.cti file? Dutch Braille with math related you use only between a and f characters the virtual dots related, or use the dot9 too?

Attila

bertfrees commented 5 months ago

I can look at it maybe in the weekend.

bertfrees commented 5 months ago

I think changing the underscore from 4569 to 456a might already fix it.

hammera commented 5 months ago

Very thank you Bert.

hammera commented 5 months ago

Bert, thanks the help! Works great (in Liblouis 3.30 only of course, Liblouis 3.29.0 not compatible the replacement). :-):-) I opened Pr #107 with future fixing this issue, but now the sanitizer check and an another check failing. Just a moment, I trying a magic thing, change the PR the Liblouis stable version number from 3.29.0 to 3.30 (I forgotted this). :-):-)

Attila

hammera commented 5 months ago

Bingo, version number increase the four affected files resolve the #107 check workflows related errors. After the #107 merge this issue I think automatically closing.

Thanks the help the fix related.

Attila

sthibaul commented 5 months ago

Unfortunatelly, very difficult to doing a rolling development Liblouis package for debian

Why so? It's a matter of somebody taking the time to upload snapshots to the experimental repository, like is done for e.g. gcc, llvm, etc.

hammera commented 5 months ago

Samuel, technical some way possible doing a day basis updated liblouis-bin-git style named package with automatically updates day by day in experimental repository when happens a change the main Liblouis repository the master branch? So, for example liblouis-bin named with liblouis-bin-git, liblouis-data named with liblouis-data-git, etc., etc., etc. Of course need handling conflicts with normal Liblouis related packages, and need handling perhaps provides and breaks fields. So, I would like similar possibility with Arch doing with Orca main branch the orca-git package in Aur. This package doing in Debian side need a Debian uploader permission, or anybody have possibility do a master branch Liblouis git based package upload the experimental repository? How can syncing always the build system from the Github repository automatically (similar with Launchpad possible doing in Ubuntu syde the package recipe the git-build-recipe way)? So, if anybody want using always the development fresh Liblouis version during development cicle with apt update method, this is a good possibility future if technical possible doing this in Debian.

Sorry the wrong english comment if difficulter understandable, but you sure understand what can I would like.

Attila

sthibaul commented 5 months ago

doing a day basis updated liblouis-bin-git

Daily uploads to experimental would be frowned upon :) In that case a PPA would be preferrable.

for example liblouis-bin named with liblouis-bin-git

I don't think we want to rename the packages (lots of reasons including package/files exclusions that will be hard to sort out). Just appending the version number with the git date should be fine.

I would like similar possibility with Arch doing with Orca main branch the orca-git package in Aur.

Similarly, a PPA with appended versions can be fine for that.

This package doing in Debian side need a Debian uploader permission,

For uploads yes, but for a PPA no.

hammera commented 5 months ago

Ok Samuel, everithing clear. In My Launchpad PPA I will doing this. In Debian codename related possible creating a PPA (for example Bookworm) in Launchpad, or I use for example with 24.04 code series in Ubuntu to provide compatibility for Bookworm? Oldest years Launchpad not provide possibility to create PPA for Debian, but possible this is already changed.

Attila

sthibaul commented 5 months ago

I don't know exactly how PPA works in ubuntu, but you can create a PPA on a mere webserver, you just need to build the package in the various targetted debian distributions, put them in separate directories, and run a few commands to generate the metadata

hammera commented 5 months ago

I known for example with Reprepro (tryed with local system), but the interesting thing how can possible doing the automatic architectures depending builds for example i386, amd64, armhf, and ARM64 (this is the four typical architecture I think with blind users using) a mere web server when the Github Liblouis repo happens a change for example the master branch. Because I not known yet this special techniques with Debian related detailed, now using normal free Launchpad PPA and Ubuntu codenames. :-):-)

Attila

egli commented 5 months ago

Fixed in #107

sthibaul commented 5 months ago

how can possible doing the automatic architectures depending builds for example i386, amd64, armhf, and ARM64

By first installing the qemu-user-static package, one can then create foreign chroots with debootstrap and run builds&tests in it. All the programs run in qemu simulation so that's much slower, but for liblouis that should be fine enough.