search

bowen-xu / PyNARS

MIT License
26 stars 8 forks source link

Kanren reasoner #94

Closed maxeeem closed 3 months ago

maxeeem commented 6 months ago

ToDo:

Notes:

github-actions[bot] commented 6 months ago

dev.txt - DEV BRANCH: FAILED (failures=54, errors=13) pr.txt - PR BRANCH: FAILED (failures=55, errors=11)

--- dev.txt 2024-05-14 20:46:21.808333256 +0000
+++ pr.txt  2024-05-14 20:46:21.812333244 +0000
@@ -1,37 +1,3 @@
-test_bug_0 (test_BUG_NAL4.TEST_BUG_NAL4)
-<bird --> animal>. %1.00;0.90% ... FAIL
-test_abduction (test_NAL1.TEST_NAL1)
-'Abduction ... ok
-test_backward_inference (test_NAL1.TEST_NAL1)
-' Backward inference ... ok
-test_conversion (test_NAL1.TEST_NAL1)
-'Conversion ... ok
-test_deduction (test_NAL1.TEST_NAL1)
-'Deduction ... ok
-test_exemplification (test_NAL1.TEST_NAL1)
-'Exemplification ... ok
-test_induction (test_NAL1.TEST_NAL1)
-'Induction ... ok
-test_revision (test_NAL1.TEST_NAL1)
-'Revision ------ ... ok
-test_wh_question_0 (test_NAL1.TEST_NAL1)
-' "wh" question ... ok
-test_wh_question_1 (test_NAL1.TEST_NAL1)
-' "wh" question ... ok
-test_yn_question (test_NAL1.TEST_NAL1)
-'"y/n" question ... ok
-test_analogy_0 (test_NAL2.TEST_NAL2)
-'Analogy ... ok
-test_analogy_1 (test_NAL2.TEST_NAL2)
-'Analogy ... FAIL
-test_backward_inference (test_NAL2.TEST_NAL2)
-'Backward inference ... ok
-test_comparison (test_NAL2.TEST_NAL2)
-'Comparison ... ok
-test_conversions_between_inheritance_and_similarity (test_NAL2.TEST_NAL2)
-'Conversions between inheritance and similarity ... ok
-test_conversions_between_inheritance_and_similarity_0 (test_NAL2.TEST_NAL2)
-'Conversions between inheritance and similarity ... FAIL
 test_conversions_between_inheritance_and_similarity_1 (test_NAL2.TEST_NAL2)
 'Conversions between inheritance and similarity ... FAIL
 test_conversions_between_inheritance_and_similarity_2 (test_NAL2.TEST_NAL2)
@@ -41,17 +7,17 @@
 test_revision (test_NAL2.TEST_NAL2)
 'Revision ... ok
 test_set_definition_0 (test_NAL2.TEST_NAL2)
-'Set definition ... FAIL
+'Set definition ... ok
 test_set_definition_1 (test_NAL2.TEST_NAL2)
 'Set definition ... FAIL
 test_set_definition_2 (test_NAL2.TEST_NAL2)
-'Set definition ... FAIL
+'Set definition ... ok
 test_set_definition_3 (test_NAL2.TEST_NAL2)
-'Set definition ... FAIL
+'Set definition ... ok
 test_structure_transformation_0 (test_NAL2.TEST_NAL2)
 'Structure transformation ... FAIL
 test_structure_transformation_1 (test_NAL2.TEST_NAL2)
-'Structure transformation ... FAIL
+'Structure transformation ... ok
 test_translating_instance_into_inheritance (test_NAL2.TEST_NAL2)
 'Translating instance into inheritance ... ok
 test_translating_instance_property_into_inheritance (test_NAL2.TEST_NAL2)
@@ -92,6 +58,7 @@
 'Boys and girls are youth. ... FAIL
 test_uni_decomposition_3 (test_NAL3.TEST_NAL3)
 'What differs boys from gials are being strong. ... FAIL
+test_other_stuff (test_NAL4.TEST_NAL4) ... ok
 test_structural_transformation_0 (test_NAL4.TEST_NAL4)
 'Structural transformation ... ok
 test_structural_transformation_1 (test_NAL4.TEST_NAL4)
@@ -126,7 +93,7 @@
 test_composition_0 (test_NAL5.TEST_NAL5)
 'compound decomposition, two premises ... FAIL
 test_composition_1 (test_NAL5.TEST_NAL5)
-'compound decomposition, two premises ... ERROR
+'compound decomposition, two premises ... ok
 test_conditional_abduction (test_NAL5.TEST_NAL5)
 'Detachment ... ok
 test_conditional_abduction_0 (test_NAL5.TEST_NAL5)
@@ -156,7 +123,7 @@
 test_conversions_between_implication_and_equivalence (test_NAL5.TEST_NAL5)
 'conversions between Implication and Equivalence ... ok
 test_decomposition_0 (test_NAL5.TEST_NAL5)
-'compound decomposition, two premises ... ok
+'compound decomposition, two premises ... FAIL
 test_decomposition_1 (test_NAL5.TEST_NAL5)
 'compound decomposition, two premises ... ok
 test_decomposition_2 (test_NAL5.TEST_NAL5)
@@ -172,9 +139,9 @@
 test_induction (test_NAL5.TEST_NAL5)
 'Induction ... ok
 test_negation_0 (test_NAL5.TEST_NAL5)
-'negation ... FAIL
+'negation ... ok
 test_negation_1 (test_NAL5.TEST_NAL5)
-'negation ... FAIL
+'negation ... ok
 test_question_0 (test_NAL5.TEST_NAL5) ... ok
 test_question_1 (test_NAL5.TEST_NAL5) ... ok
 test_resemblance (test_NAL5.TEST_NAL5)
@@ -182,33 +149,29 @@
 test_revision (test_NAL5.TEST_NAL5)
 'Revision ... ok
 test_0 (test_NAL6.TEST_NAL6)
-<(&&, <#x-->A>, <#x-->B>, <<$y-->C>==><$y-->D>>, <$z-->E>) ==> (&&, <$z-->F>, <#p-->G>, <#p-->H>)>. ... ERROR
+<(&&, <#x-->A>, <#x-->B>, <<$y-->C>==><$y-->D>>, <$z-->E>) ==> (&&, <$z-->F>, <#p-->G>, <#p-->H>)>. ... FAIL
 test_1 (test_NAL6.TEST_NAL6)
 (&&, <#x-->A>, <#x-->B>, <<$y-->C>==><$y-->D>>). ... FAIL
 test_abduction_with_variable_elimination_abduction (test_NAL6.TEST_NAL6)
 'Abduction with variable elimination (abduction) ... ok
 test_abduction_with_variable_elimination_abduction_0 (test_NAL6.TEST_NAL6)
 <<M --> A> ==> C>. %1.00;0.90% ... ok
-test_abduction_with_variable_elimination_abduction_1 (test_NAL6.TEST_NAL6)
-<<M --> A> ==> C>. %1.00;0.90% ... ok
 test_birdClaimedByBob (test_NAL6.TEST_NAL6)
 'from https://code.google.com/archive/p/open-nars/issues/7 ... ok
 test_can_of_worms (test_NAL6.TEST_NAL6)
 <0 --> num>. %1.00;0.90% ... ok
 test_elimination_0 (test_NAL6.TEST_NAL6)
-'Variable elimination ... FAIL
+'Variable elimination ... ok
 test_elimination_1 (test_NAL6.TEST_NAL6)
-'Variable elimination ... FAIL
+'Variable elimination ... ok
 test_elimination_2 (test_NAL6.TEST_NAL6)
-'Variable elimination ... FAIL
-test_elimination_3 (test_NAL6.TEST_NAL6)
 'Variable elimination ... ok
-test_elimination_3_1 (test_NAL6.TEST_NAL6)
-(&&,<C --> A>,<D --> B>). %1.00;0.90% ... ok
-test_elimination_4 (test_NAL6.TEST_NAL6)
+test_elimination_3 (test_NAL6.TEST_NAL6)
 'Variable elimination ... FAIL
+test_elimination_4 (test_NAL6.TEST_NAL6)
+'Variable elimination ... ok
 test_elimination_5 (test_NAL6.TEST_NAL6)
-'Variable elimination ... FAIL
+'Variable elimination ... ok
 test_elimination_6 (test_NAL6.TEST_NAL6)
 'Variable elimination ... FAIL
 test_introduction_0 (test_NAL6.TEST_NAL6)
@@ -222,9 +185,9 @@
 test_multiple_variable_elimination_1 (test_NAL6.TEST_NAL6)
 'Multiple variable elimination ... ok
 test_multiple_variable_elimination_2 (test_NAL6.TEST_NAL6)
-'Multiple variable elimination ... ok
+'Multiple variable elimination ... FAIL
 test_multiple_variables_introduction_0 (test_NAL6.TEST_NAL6)
-'Multiple variables introduction ... FAIL
+'Multiple variables introduction ... ok
 test_multiple_variables_introduction_1 (test_NAL6.TEST_NAL6)
 'Multiple variables introduction ... ok
 test_nlp1 (test_NAL6.TEST_NAL6)
@@ -241,10 +204,8 @@
 'Second level variable unification ... ok
 test_second_level_variable_unification_1_0 (test_NAL6.TEST_NAL6)
 <A ==> (&&,<#2 --> B>,C)>. %1.00;0.90% ... ok
-test_second_level_variable_unification_1_1 (test_NAL6.TEST_NAL6)
-<A ==> (&&,<D --> B>,C)>. %1.00;0.90% ... ok
 test_second_variable_introduction_induction (test_NAL6.TEST_NAL6)
-'Second variable introduction (induction) ... ok
+'Second variable introduction (induction) ... FAIL
 test_symmetry (test_NAL6.TEST_NAL6)
 <(*,a,b) --> like>. %1.00;0.90% ... ok
 test_uncle (test_NAL6.TEST_NAL6)
@@ -271,12 +232,11 @@
 'Second variable introduction (induction) ... ok
 test_variable_elimination_deduction_0 (test_NAL6.TEST_NAL6)
 <M --> A>. %1.00;0.90% ... ok
-test_variable_elimination_deduction_1 (test_NAL6.TEST_NAL6)
-<M --> A>. %1.00;0.90% ... ok
+test_variables (test_NAL6.TEST_NAL6) ... FAIL
 test_abduction (test_NAL7.TEST_NAL7)
 'Temporal abudction ... ok
 test_abduction_sequence_eliminate_0 (test_NAL7.TEST_NAL7)
-nal7.19.nal ... ok
+nal7.19.nal ... FAIL
 test_abduction_sequence_eliminate_1 (test_NAL7.TEST_NAL7)
 nal7.19.nal ... FAIL
 test_analogy (test_NAL7.TEST_NAL7)
@@ -292,7 +252,7 @@
 test_deduction (test_NAL7.TEST_NAL7)
 'Temporal deduction ... ok
 test_deduction_sequence (test_NAL7.TEST_NAL7)
-nal7.36.nal ... ok
+nal7.36.nal ... FAIL
 test_deduction_sequence_eliminate_0 (test_NAL7.TEST_NAL7)
 nal7.18.nal ... ERROR
 test_deduction_sequence_eliminate_0_1 (test_NAL7.TEST_NAL7) ... ERROR
@@ -313,37 +273,37 @@
 test_inference_on_tense_0 (test_NAL7.TEST_NAL7)
 'Inference on tense ... FAIL
 test_inference_on_tense_1 (test_NAL7.TEST_NAL7)
-' inference on tense ... FAIL
+' inference on tense ... ERROR
 test_inference_on_tense_2 (test_NAL7.TEST_NAL7)
 ' inference on tense ... ok
-test_analogy_0_0__0 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_0_0__0 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
 test_analogy_0_0__1 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_analogy_0_0__2 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_analogy_0_0__3 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_0_0__4 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_0_0__4 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
 test_analogy_0_0__5 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_0_1__0 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_0_1__0 (test_NAL7.TEST_NAL7_ANALOGY) ... ERROR
 test_analogy_0_1__1 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_0_1__2 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_0_1__2 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
 test_analogy_0_1__3 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_analogy_0_1__4 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_analogy_0_1__5 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_analogy_1_0__0 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_analogy_1_0__1 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_1_0__2 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_1_0__2 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
 test_analogy_1_0__3 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_analogy_1_0__4 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_analogy_1_0__5 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_1_1__0 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
+test_analogy_1_1__0 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_analogy_1_1__1 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_analogy_1_1__2 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_analogy_1_1__3 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_1_1__4 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_1_1__4 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
 test_analogy_1_1__5 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
 test_1_0 (test_NAL8.TEST_NAL8)
 nal8.1.0.nal ... ok
 test_1_1 (test_NAL8.TEST_NAL8)
-nal8.1.1.nal ... FAIL
+nal8.1.1.nal ... ok
 test_1_10 (test_NAL8.TEST_NAL8)
 nal8.1.10.nal ... ERROR
 test_1_11 (test_NAL8.TEST_NAL8)
@@ -353,21 +313,21 @@
 test_1_13_var (test_NAL8.TEST_NAL8)
 nal8.1.13.nal ... ERROR
 test_1_14 (test_NAL8.TEST_NAL8)
-nal8.1.14.nal ... ok
+nal8.1.14.nal ... FAIL
 test_1_16 (test_NAL8.TEST_NAL8)
 nal8.1.16.nal ... FAIL
 test_1_2 (test_NAL8.TEST_NAL8)
 nal8.1.2.nal ... FAIL
 test_1_3 (test_NAL8.TEST_NAL8)
-nal8.1.3.nal ... ok
+nal8.1.3.nal ... FAIL
 test_1_3_var (test_NAL8.TEST_NAL8)
-nal8.1.3.nal ... ERROR
+nal8.1.3.nal ... FAIL
 test_1_4 (test_NAL8.TEST_NAL8)
-nal8.1.4.nal ... ok
+nal8.1.4.nal ... FAIL
 test_1_4_var (test_NAL8.TEST_NAL8)
 nal8.1.4.nal ... FAIL
 test_1_5 (test_NAL8.TEST_NAL8)
-nal8.1.5.nal ... ERROR
+nal8.1.5.nal ... ok
 test_1_7 (test_NAL8.TEST_NAL8)
 nal8.1.7.nal ... FAIL
 test_1_7_var (test_NAL8.TEST_NAL8)
@@ -377,7 +337,7 @@
 test_1_9 (test_NAL8.TEST_NAL8)
 nal8.1.9.nal ... FAIL
 test_sequence_0 (test_NAL8.TEST_NAL8)
-(&/, A, B, C). ... ok
+(&/, A, B, C). ... FAIL
 test_sequence_1 (test_NAL8.TEST_NAL8)
 C! ... FAIL
 test_sequence_2 (test_NAL8.TEST_NAL8)
maxeeem commented 4 months ago

@bowen-xu @ccrock4t So I discovered a problem with swapping out inference engines. I can pass the parameter as we discussed and use either GeneralInference or KanrenEngine and this works. But the problem is that since there are differences in implementation, for example how negation is handled, and some test cases were changed to explicitly use -- instead of frequency of 0. So these test cases will fail if used with GeneralInference engine even thought they work fine on the dev branch. This also applies to some truth values. And anything that relies on truth_analytic being 0.9,0.9 instead of 1.0,1,0.

So I'm looking for your input here. What should we do about it? Technically the GeneralInference still works the same but a lot more test cases fail on this branch compared to the dev branch because the test cases themselves have been changed.

ccrock4t commented 4 months ago

How many cases are failing due to this? At any rate, I guess the new tests / KanrenEngine should take priority. Though, would it be a big or small change to update the GeneralEngine to use negation instead?

maxeeem commented 4 months ago

How many cases are failing due to this? At any rate, I guess the new tests / KanrenEngine should take priority. Though, would it be a big or small change to update the GeneralEngine to use negation instead?

Due to negation probably just a few. Most are because of the different truth values I believe.

maxeeem commented 4 months ago

@ccrock4t @bowen-xu Also, here's a fun little comparison of performance between Kanren and General inference engines. https://youtu.be/07NTJhEkiVQ

I guess the takeaway is that there's no easy way to compare the two directly and there is room to improve the computation in KanrenEngine but it is not directly related to miniKanren, which is performant as is, but rather to the way immediate and structural inference are handled.

bowen-xu commented 3 months ago

I understand Inference Engine is quite a hard work... Thank you, Maxim!