Segmentation Fault `lazy_findall ` probably related to Undefined Hook Predicate in Multithreaded Environment

TeamSPoon commented 3 weeks ago

There is a segmentation fault occurring in MeTTaLog when executing certain queries that involve lazy evaluation with lazy_findall/3 in a multithreaded environment. It seems that a hook predicate might be undefined across different threads, leading to the crash when Prolog attempts to complain or raise an error, particularly in a message_hook.

Reproduction Steps:

Run the following command using MeTTaLog:
```
mettalog --prolog
```
Execute the following query:

?-  lazy_findall(Res1,between(1,10,Res1),List1),
     lazy_findall(Res2,between(5,10,Res2),List2),
     member(E1,List1),member(E2,List2),E1=E2.

The query triggers a segmentation fault with the following output:

ERROR: Received fatal signal 11 (segv)
Time: Mon Aug 19 14:12:56 2024
Inferences: 22492715
Thread: 1 (main)
C-stack trace labeled "crash":
  [0] save_backtrace() at /opt/hyperon/metta-wam/swipl-devel/src/os/pl-cstack.c:337 [0x79cf798649b5]
  [1] sigCrashHandler() at /opt/hyperon/metta-wam/swipl-devel/src/os/pl-cstack.c:937 [0x79cf79864a9d]
  [2] __restore_rt() at libc_sigaction.c:? [0x79cf79442520]
  [3] ___pthread_mutex_lock() at ./nptl/pthread_mutex_lock.c:80 [0x79cf79497ef4]
  [4] interactor_post_answer___LD() at /opt/hyperon/metta-wam/swipl-devel/src/pl-thread.c:4049 [0x79cf797e903e]
  [5] pl_engine_next2_va() at /opt/hyperon/metta-wam/swipl-devel/src/pl-thread.c:4069 [0x79cf797e94dc]
  [6] PL_next_solution___LD() at /opt/hyperon/metta-wam/swipl-devel/src/pl-vmi.c:4479 [0x79cf79762026]
  [7] query_loop() at /opt/hyperon/metta-wam/swipl-devel/src/pl-pro.c:155 [0x79cf797aa7d2]
  [8] pl_break1() at /opt/hyperon/metta-wam/swipl-devel/src/pl-pro.c:224 [0x79cf797aaa67]
  [9] PL_next_solution___LD() at /opt/hyperon/metta-wam/swipl-devel/src/pl-vmi.c:4495 [0x79cf79761f6c]
  [10] callProlog() at /opt/hyperon/metta-wam/swipl-devel/src/pl-pro.c:502 [0x79cf797aad71]
  [11] pl_sig_atomic1_va() at /opt/hyperon/metta-wam/swipl-devel/src/pl-pro.c:394 [0x79cf797aaf67]
  [12] PL_next_solution___LD() at /opt/hyperon/metta-wam/swipl-devel/src/pl-vmi.c:4479 [0x79cf79762026]
  [13] query_loop() at /opt/hyperon/metta-wam/swipl-devel/src/pl-pro.c:155 [0x79cf797aa7d2]
  [14] prologToplevel() at /opt/hyperon/metta-wam/swipl-devel/src/pl-pro.c:594 [0x79cf797ab151]
  [15] swipl(+0x10eb) [0x58b08c65b0eb]
  [16] __libc_start_call_main() at ./csu/../sysdeps/nptl/libc_start_call_main.h:58 [0x79cf79429d90]
  [17] call_init() at ./csu/../csu/libc-start.c:128 [0x79cf79429e40]
  [18] swipl(+0x1135) [0x58b08c65b135]

PROLOG STACK:
     [69] [system] engine_next(<engine>(3,0x58b0912c6910), _202394)
     [68] [lazy_lists] lazy_engine_next(<engine>(3,0x58b0912c6910), 1, _202388, _202390)
     [67] [lazy_lists] attr_unify_hook(lazy_list(lazy_lists:lazy_engine_next(<engine>(3,0x58b0912c6910), 1), _202296), [_202360|_202362])
     [66] [$attvar] uhook(lazy_lists, lazy_list(lazy_lists:lazy_engine_next(<engine>(3,0x58b0912c6910), 1), _202296), [_202360|_202362])
     [65] [$attvar] call_all_attr_uhooks(att(lazy_lists, lazy_list(lazy_lists:lazy_engine_next(<engine>(3,0x58b0912c6910), 1), _202296), []), [_202360|_202362])
     [64] [$attvar] '$wakeup'(wakeup(att(lazy_lists, lazy_list(lazy_lists:lazy_engine_next(<engine>(3,0x58b0912c6910), 1), _202296), []), [_202360|_202362], []))
     [63] [lists] lists:member(_200978, [_202360|_202362])
     [62] [user] '<meta-call>'(user:user:(lazy_findall(_200950, between(1, 10, _200950), [_202360|_202362]), lazy_findall(_200966, between(5, 10, _200966), _202346{lazy_lists = ...}), member(_200978, [_202360|_202362]), member(_200984, _202346{lazy_lists = ...}), _200978=_200984))
     [61] [$toplevel] toplevel_call(user:user:(lazy_findall(_200950, between(1, 10, _200950), [_202360|_202362]), lazy_findall(_200966, between(5, 10, _200966), _202346{lazy_lists = ...}), member(_200978, [_202360|_202362]), member(_200984, _202346{lazy_lists = ...}), _200978=_200984))
     [60] [$toplevel] stop_backtrace(user:user:(lazy_findall(_200950, between(1, 10, _200950), [_202360|_202362]), lazy_findall(_200966, between(5, 10, _200966), _202346{lazy_lists = ...}), member(_200978, [_202360|_202362]), member(_200984, _202346{lazy_lists = ...}), _200978=_200984), _202248)
Running on_halt hooks with status 139
Killing 315018 with default signal handlers
/opt/hyperon/metta-wam/mettalog: line 201: 315018 Segmentation fault      (core dumped) swipl -l /opt/hyperon/metta-wam/src/canary/metta_interp.pl -- --python=enable --prolog

Observations:

The issue might be related to an undefined hook predicate being called across different threads.
The crash appears to occur when Prolog attempts to generate an error message via message_hook but fails due to the missing predicate.
Running the same query directly in SWI-Prolog (without MeTTaLog) completes successfully without any crash.

Additional Information:

SWI-Prolog version: 9.3.7-47-g1ff28598f-DIRTY (development build).
The issue is reproducible consistently when the query involves lazy engines created via lazy_findall/3.
Using a different environment without the custom hooks used by MeTTaLog does not produce the segmentation fault.

Expected Behavior:

The expected behavior for this query when running directly in SWI-Prolog (without MeTTaLog) is as follows:

?-  lazy_findall(Res1,between(1,10,Res1),List1),
     lazy_findall(Res2,between(5,10,Res2),List2),
     member(E1,List1),member(E2,List2),E1=E2.

List1 = [1, 2, 3, 4, 5|_A],
List2 = [5|_B],
E1 = E2, E2 = 5,
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(3,0x58db76358a00), 1), _A) ;
List1 = [1, 2, 3, 4, 5, 6|_A],
List2 = [5, 6|_B],
E1 = E2, E2 = 6,
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(3,0x58db76358a00), 1), _A) ;
List1 = [1, 2, 3, 4, 5, 6, 7|_A],
List2 = [5, 6, 7|_B],
E1 = E2, E2 = 7,
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(3,0x58db76358a00), 1), _A) ;
List1 = [1, 2, 3, 4, 5, 6, 7, 8|_A],
List2 = [5, 6, 7, 8|_B],
E1 = E2, E2 = 8,
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(3,0x58db76358a00), 1), _A) ;
List1 = [1, 2, 3, 4, 5, 6, 7, 8, 9|...],
List2 = [5, 6, 7, 8, 9|_B],
E1 = E2, E2 = 9,
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(3,0x58db76358a00), 1), _A) ;
List1 = [1, 2, 3, 4, 5, 6, 7, 8, 9|...],
List2 = [5, 6, 7, 8, 9, 10|_B],
E1 = E2, E2 = 10,
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x58db76358a80), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(3,0x58db76358a00), 1), _A) ;
false.

?-

Actual Behavior:

The system crashes with a segmentation fault, making it impossible to diagnose the root cause through conventional error reporting mechanisms.

Suggested Fixes or Investigation Areas:

Investigate whether a hook predicate is being called in a different thread context and ensure that it is defined and accessible.
Enhance the robustness of message_hook handling to prevent crashes when an error message cannot be delivered.
Consider logging missing hook predicates to help track down the source of the issue.

TeamSPoon commented 1 week ago

Fixed


deb12user@GODLIKE:~$ mettalog
metta+>prolog.
[1 ?-
|
|     lazy_findall(Res1,between(1,10,Res1),List1),
|         lazy_findall(Res2,between(5,10,Res2),List2),
|         member(E1,List1),member(E2,List2),E1=E2.
List1 = [1, 2, 3, 4, 5|_A],
List2 = [5|_B],
E1 = E2, E2 = 5,
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x5610945e0860), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x5610945e0860), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x5610945e0860), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x5610945e0860), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x5610945e0860), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x5610945e0860), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(3,0x561094e54250), 1), _A) .

TeamSPoon commented 1 week ago

well

2 ?-
|    lazy_findall(Res1,between(1,10,Res1),List1),
     lazy_findall(Res2,between(5,10,Res2),List2),
     member(E1,List1),member(E2,List2),E1=E2.
List1 = [1, 2, 3, 4, 5|_A],
List2 = [5|_B],
E1 = E2, E2 = 5,
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x561094c1e650), 1), _A) ;
List1 = [1, 2, 3, 4, 5, 6|_A],
List2 = [5, 6|_B],
E1 = E2, E2 = 6,
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x561094c1e650), 1), _A) ;
List1 = [1, 2, 3, 4, 5, 6, 7|_A],
List2 = [5, 6, 7|_B],
E1 = E2, E2 = 7,
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x561094c1e650), 1), _A) ;
List1 = [1, 2, 3, 4, 5, 6, 7, 8|_A],
List2 = [5, 6, 7, 8|_B],
E1 = E2, E2 = 8,
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x561094c1e650), 1), _A) ;
List1 = [1, 2, 3, 4, 5, 6, 7, 8, 9|_A],
List2 = [5, 6, 7, 8, 9|_B],
E1 = E2, E2 = 9,
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x561094c1e650), 1), _A) ;
List1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10|_A],
List2 = [5, 6, 7, 8, 9, 10|_B],
E1 = E2, E2 = 10,
lazy_list(lazy_lists:lazy_engine_next(<engine>(5,0x561094cbc1e0), 1), _B),
lazy_list(lazy_lists:lazy_engine_next(<engine>(4,0x561094c1e650), 1), _A) ;
false.

trueagi-io / metta-wam

Segmentation Fault `lazy_findall ` probably related to Undefined Hook Predicate in Multithreaded Environment #92