catch/3 fails to catch exception

[pmoura]

I recently added some more tests for the Prolog standard bagof/3 and setof/3 predicates to the current Logtalk git version. With the latest GNU Prolog git version (), I get:

$ logtalk_tester -p gnu
% Batch testing started @ 2023-03-22 18:39:06
%         Logtalk version: 3.64.0-b01
%         GNU Prolog version: 1.5.1
%
% logtalk/tests/prolog/predicates/bagof_3
%         23 tests: 1 skipped, 21 passed, 1 failed (0 flaky)
%         completed tests from object tests in 3 seconds
%         clause coverage n/a
%
% Compilation errors/warnings and failed unit tests
% (compilation errors/warnings might be expected depending on the test)
!     lgt_bagof_3_23: failure (in 0.0 seconds)
!       test goal failed but should have thrown an error:
!         expected error(existence_error(procedure,foobar/1),A)
!       in file logtalk/tests/prolog/predicates/bagof_3/tests.lgt between lines 169-170
%
% Skipped tests
logtalk/tests/prolog/predicates/bagof_3/tests.lgt - lgt_bagof_3_22 @ tests
%
% Failed tests
logtalk/tests/prolog/predicates/bagof_3/tests.lgt - lgt_bagof_3_23 @ tests
%
% 1 test sets: 1 completed, 0 skipped, 0 broken, 0 timedout, 0 crashed
% 23 tests: 1 skipped, 21 passed, 1 failed (0 flaky)
%
% Batch testing ended @ 2023-03-22 18:39:10

Similar for setof/3. The failing test is:

test(lgt_bagof_3_23, errors([existence_error(procedure,foobar/1), existence_error(procedure,':'(user,foobar/1))])) :-
    {bagof(X, foobar(X), _)}.

If we try the goal at the top-level, we do get the expected exception:

| ?- bagof(X, foobar(X), _).
uncaught exception: error(existence_error(procedure,foobar/1),bagof/3)

The tests are run using a catch/3 goal that catches all exceptions. Thus, this may be a dereferencing bug?

[pmoura]

If I trace the test execution:

| ?- tests::run(lgt_setof_3_37).
   Call: (1) tests::run(lgt_setof_3_37) ? 
   Call: (2) test(lgt_setof_3_37,_9760) ? 
   Fact: (2) test(lgt_setof_3_37,throws(lgt_setof_3_37,[],[error(existence_error(procedure,foobar/1),_12122),error(existence_error(procedure,user:foobar/1),_12133)],295-296)) ? 
   Exit: (2) test(lgt_setof_3_37,throws(lgt_setof_3_37,[],[error(existence_error(procedure,foobar/1),_12122),error(existence_error(procedure,user:foobar/1),_12133)],295-296)) ? 
   Call: (3) test(lgt_setof_3_37,[],[error(existence_error(procedure,foobar/1),_12122),error(existence_error(procedure,user:foobar/1),_12133)]) ? 
   Rule: (3) test(lgt_setof_3_37,[],[error(existence_error(procedure,foobar/1),_12122),error(existence_error(procedure,user:foobar/1),_12133)]) ? 
   Call: (4) {setof(_18564,foobar(_18564),_18566)} ? 
     Compiled goal: setof(A,foobar(A),B)
   Call: (4) {setof(_18564,foobar(_18564),_18566)} ? 
   Fail: (4) {setof(_18564,foobar(_18564),_18566)} ? 
   Fail: (3) test(lgt_setof_3_37,[],[error(existence_error(procedure,foobar/1),_12122),error(existence_error(procedure,user:foobar/1),_12133)]) ? 
!     lgt_setof_3_37: failure (in 0.0010000000000001119 seconds)
!       test goal failed but should have thrown an error:
!         expected error(existence_error(procedure,foobar/1),A)
!       in file /Users/pmoura/logtalk/tests/prolog/predicates/setof_3/tests.lgt between lines 295-296
...

The "Compiled goal:" line shows the actual goal being called ({}/1 is a Logtalk compiler bypass control construct, used here to ensure that the test goal is called as-is).

[didoudiaz]

Is it a new bug (or present in previous version) ?

[pmoura]

No idea. I didn’t try these new tests with 1.5.0.

[pmoura]

If we break during the debugging session:

| ?- tests::run(lgt_setof_3_37).
   Call: (1) tests::run(lgt_setof_3_37) ? 
   Call: (2) test(lgt_setof_3_37,_9760) ? 
   Fact: (2) test(lgt_setof_3_37,throws(lgt_setof_3_37,[],[error(existence_error(procedure,foobar/1),_12122),error(existence_error(procedure,user:foobar/1),_12133)],295-296)) ? 
   Exit: (2) test(lgt_setof_3_37,throws(lgt_setof_3_37,[],[error(existence_error(procedure,foobar/1),_12122),error(existence_error(procedure,user:foobar/1),_12133)],295-296)) ? 
   Call: (3) test(lgt_setof_3_37,[],[error(existence_error(procedure,foobar/1),_12122),error(existence_error(procedure,user:foobar/1),_12133)]) ? 
   Rule: (3) test(lgt_setof_3_37,[],[error(existence_error(procedure,foobar/1),_12122),error(existence_error(procedure,user:foobar/1),_12133)]) ? 
   Call: (4) {setof(_18564,foobar(_18564),_18566)} ? 
   Fail: (4) {setof(_18564,foobar(_18564),_18566)} ? b
{Break Level 1}
{1}
| ?- setof(A,foobar(A),B).

no
{1}
| ?- setof(A,foobar42(A),B).

no
{1}
| ?- 

[didoudiaz]

Somewhere the Prolog flag unknown is set to fail while running{tester}, thus the problem:

$ gplgt.sh 
GNU Prolog 1.5.1 (64 bits)
Compiled Mar 22 2023, 18:39:20 with gcc
Copyright (C) 1999-2023 Daniel Diaz
...
Logtalk 3.64.0-b01
Copyright (c) 1998-2023 Paulo Moura
...
% For Logtalk compiler warnings/errors explanations and fix suggestions, use:
% ?- {tutor(loader)}. or ?- logtalk_load(tutor(loader)).
% 
| ?- current_prolog_flag(unknown, X).

X = error

yes
| ?- {tester}.                       
...% 
% tests started at 2023-03-22, 23:20:45
% 
% running tests from object tests
% file: /home/diaz/lgt3git/tests/prolog/predicates/setof_3/tests.lgt
...
% lgt_setof_3_36: skipped
!     lgt_setof_3_37: failure (in 0.0 seconds)
!       test goal failed but should have thrown an error:
!         expected error(existence_error(procedure,foobar/1),A)
!       in file /home/diaz/lgt3git/tests/prolog/predicates/setof_3/tests.lgt between lines 295-296
% wg17_setof_3_38: success (in 0.0 seconds)
% 
% 38 tests: 2 skipped, 35 passed, 1 failed (0 flaky)
% completed tests from object tests

| ?- current_prolog_flag(unknown, X).

X = fail

yes
| ?- set_prolog_flag(unknown, error).

yes
| ?- tests::run(lgt_setof_3_37).
% lgt_setof_3_37: success (in 0.0 seconds)
% 
% 1 tests: 0 skipped, 1 passed, 0 failed (0 flaky)
% completed tests from object tests
% 
% no code coverage information collected

BTW: can I "load" the tests without executing all (as done by {tester} and only run one test, e.g. tests::run(lgt_setof_3_37) ?

[pmoura]

Odd. Logtalk never changes that flag.

[pmoura]

You can modify the tester.lgt file and change the tests::run goal to tests::run(lgt_setof_3_37).

[didoudiaz]

Odd. Logtalk never changes that flag.

Rapidly doing a grep in the git repo:

...
grep -nr 'set_prolog_flag( *unknown' .
tests/prolog/predicates/setof_3/tests.lgt:137:          {set_prolog_flag(unknown, fail),
tests/prolog/predicates/bagof_3/tests.lgt:96:           {set_prolog_flag(unknown, fail),

[pmoura]

Oops! The tests indeed change the flag! Look at the iso_bagof_3_09 and iso_setof_3_11 tests. Those tests are inside a conditional compilation block that, in the case of GNU Prolog (but not for the other systems where I run the new tests), results in the "else" definition to run. This happens due to GNU Prolog not defining (^)/1 a predicate (which is perfectly fine). I will fix the tests. Sorry for the false alarm and wasting your time like this!

[pmoura]

Fixed tests pushed. All bagof/3 and setof/3 tests now pass.