Closed maxeeem closed 3 months ago
dev.txt - DEV BRANCH: FAILED (failures=54, errors=13) pr.txt - PR BRANCH: FAILED (failures=55, errors=11)
--- dev.txt 2024-05-14 20:46:21.808333256 +0000
+++ pr.txt 2024-05-14 20:46:21.812333244 +0000
@@ -1,37 +1,3 @@
-test_bug_0 (test_BUG_NAL4.TEST_BUG_NAL4)
-<bird --> animal>. %1.00;0.90% ... FAIL
-test_abduction (test_NAL1.TEST_NAL1)
-'Abduction ... ok
-test_backward_inference (test_NAL1.TEST_NAL1)
-' Backward inference ... ok
-test_conversion (test_NAL1.TEST_NAL1)
-'Conversion ... ok
-test_deduction (test_NAL1.TEST_NAL1)
-'Deduction ... ok
-test_exemplification (test_NAL1.TEST_NAL1)
-'Exemplification ... ok
-test_induction (test_NAL1.TEST_NAL1)
-'Induction ... ok
-test_revision (test_NAL1.TEST_NAL1)
-'Revision ------ ... ok
-test_wh_question_0 (test_NAL1.TEST_NAL1)
-' "wh" question ... ok
-test_wh_question_1 (test_NAL1.TEST_NAL1)
-' "wh" question ... ok
-test_yn_question (test_NAL1.TEST_NAL1)
-'"y/n" question ... ok
-test_analogy_0 (test_NAL2.TEST_NAL2)
-'Analogy ... ok
-test_analogy_1 (test_NAL2.TEST_NAL2)
-'Analogy ... FAIL
-test_backward_inference (test_NAL2.TEST_NAL2)
-'Backward inference ... ok
-test_comparison (test_NAL2.TEST_NAL2)
-'Comparison ... ok
-test_conversions_between_inheritance_and_similarity (test_NAL2.TEST_NAL2)
-'Conversions between inheritance and similarity ... ok
-test_conversions_between_inheritance_and_similarity_0 (test_NAL2.TEST_NAL2)
-'Conversions between inheritance and similarity ... FAIL
test_conversions_between_inheritance_and_similarity_1 (test_NAL2.TEST_NAL2)
'Conversions between inheritance and similarity ... FAIL
test_conversions_between_inheritance_and_similarity_2 (test_NAL2.TEST_NAL2)
@@ -41,17 +7,17 @@
test_revision (test_NAL2.TEST_NAL2)
'Revision ... ok
test_set_definition_0 (test_NAL2.TEST_NAL2)
-'Set definition ... FAIL
+'Set definition ... ok
test_set_definition_1 (test_NAL2.TEST_NAL2)
'Set definition ... FAIL
test_set_definition_2 (test_NAL2.TEST_NAL2)
-'Set definition ... FAIL
+'Set definition ... ok
test_set_definition_3 (test_NAL2.TEST_NAL2)
-'Set definition ... FAIL
+'Set definition ... ok
test_structure_transformation_0 (test_NAL2.TEST_NAL2)
'Structure transformation ... FAIL
test_structure_transformation_1 (test_NAL2.TEST_NAL2)
-'Structure transformation ... FAIL
+'Structure transformation ... ok
test_translating_instance_into_inheritance (test_NAL2.TEST_NAL2)
'Translating instance into inheritance ... ok
test_translating_instance_property_into_inheritance (test_NAL2.TEST_NAL2)
@@ -92,6 +58,7 @@
'Boys and girls are youth. ... FAIL
test_uni_decomposition_3 (test_NAL3.TEST_NAL3)
'What differs boys from gials are being strong. ... FAIL
+test_other_stuff (test_NAL4.TEST_NAL4) ... ok
test_structural_transformation_0 (test_NAL4.TEST_NAL4)
'Structural transformation ... ok
test_structural_transformation_1 (test_NAL4.TEST_NAL4)
@@ -126,7 +93,7 @@
test_composition_0 (test_NAL5.TEST_NAL5)
'compound decomposition, two premises ... FAIL
test_composition_1 (test_NAL5.TEST_NAL5)
-'compound decomposition, two premises ... ERROR
+'compound decomposition, two premises ... ok
test_conditional_abduction (test_NAL5.TEST_NAL5)
'Detachment ... ok
test_conditional_abduction_0 (test_NAL5.TEST_NAL5)
@@ -156,7 +123,7 @@
test_conversions_between_implication_and_equivalence (test_NAL5.TEST_NAL5)
'conversions between Implication and Equivalence ... ok
test_decomposition_0 (test_NAL5.TEST_NAL5)
-'compound decomposition, two premises ... ok
+'compound decomposition, two premises ... FAIL
test_decomposition_1 (test_NAL5.TEST_NAL5)
'compound decomposition, two premises ... ok
test_decomposition_2 (test_NAL5.TEST_NAL5)
@@ -172,9 +139,9 @@
test_induction (test_NAL5.TEST_NAL5)
'Induction ... ok
test_negation_0 (test_NAL5.TEST_NAL5)
-'negation ... FAIL
+'negation ... ok
test_negation_1 (test_NAL5.TEST_NAL5)
-'negation ... FAIL
+'negation ... ok
test_question_0 (test_NAL5.TEST_NAL5) ... ok
test_question_1 (test_NAL5.TEST_NAL5) ... ok
test_resemblance (test_NAL5.TEST_NAL5)
@@ -182,33 +149,29 @@
test_revision (test_NAL5.TEST_NAL5)
'Revision ... ok
test_0 (test_NAL6.TEST_NAL6)
-<(&&, <#x-->A>, <#x-->B>, <<$y-->C>==><$y-->D>>, <$z-->E>) ==> (&&, <$z-->F>, <#p-->G>, <#p-->H>)>. ... ERROR
+<(&&, <#x-->A>, <#x-->B>, <<$y-->C>==><$y-->D>>, <$z-->E>) ==> (&&, <$z-->F>, <#p-->G>, <#p-->H>)>. ... FAIL
test_1 (test_NAL6.TEST_NAL6)
(&&, <#x-->A>, <#x-->B>, <<$y-->C>==><$y-->D>>). ... FAIL
test_abduction_with_variable_elimination_abduction (test_NAL6.TEST_NAL6)
'Abduction with variable elimination (abduction) ... ok
test_abduction_with_variable_elimination_abduction_0 (test_NAL6.TEST_NAL6)
<<M --> A> ==> C>. %1.00;0.90% ... ok
-test_abduction_with_variable_elimination_abduction_1 (test_NAL6.TEST_NAL6)
-<<M --> A> ==> C>. %1.00;0.90% ... ok
test_birdClaimedByBob (test_NAL6.TEST_NAL6)
'from https://code.google.com/archive/p/open-nars/issues/7 ... ok
test_can_of_worms (test_NAL6.TEST_NAL6)
<0 --> num>. %1.00;0.90% ... ok
test_elimination_0 (test_NAL6.TEST_NAL6)
-'Variable elimination ... FAIL
+'Variable elimination ... ok
test_elimination_1 (test_NAL6.TEST_NAL6)
-'Variable elimination ... FAIL
+'Variable elimination ... ok
test_elimination_2 (test_NAL6.TEST_NAL6)
-'Variable elimination ... FAIL
-test_elimination_3 (test_NAL6.TEST_NAL6)
'Variable elimination ... ok
-test_elimination_3_1 (test_NAL6.TEST_NAL6)
-(&&,<C --> A>,<D --> B>). %1.00;0.90% ... ok
-test_elimination_4 (test_NAL6.TEST_NAL6)
+test_elimination_3 (test_NAL6.TEST_NAL6)
'Variable elimination ... FAIL
+test_elimination_4 (test_NAL6.TEST_NAL6)
+'Variable elimination ... ok
test_elimination_5 (test_NAL6.TEST_NAL6)
-'Variable elimination ... FAIL
+'Variable elimination ... ok
test_elimination_6 (test_NAL6.TEST_NAL6)
'Variable elimination ... FAIL
test_introduction_0 (test_NAL6.TEST_NAL6)
@@ -222,9 +185,9 @@
test_multiple_variable_elimination_1 (test_NAL6.TEST_NAL6)
'Multiple variable elimination ... ok
test_multiple_variable_elimination_2 (test_NAL6.TEST_NAL6)
-'Multiple variable elimination ... ok
+'Multiple variable elimination ... FAIL
test_multiple_variables_introduction_0 (test_NAL6.TEST_NAL6)
-'Multiple variables introduction ... FAIL
+'Multiple variables introduction ... ok
test_multiple_variables_introduction_1 (test_NAL6.TEST_NAL6)
'Multiple variables introduction ... ok
test_nlp1 (test_NAL6.TEST_NAL6)
@@ -241,10 +204,8 @@
'Second level variable unification ... ok
test_second_level_variable_unification_1_0 (test_NAL6.TEST_NAL6)
<A ==> (&&,<#2 --> B>,C)>. %1.00;0.90% ... ok
-test_second_level_variable_unification_1_1 (test_NAL6.TEST_NAL6)
-<A ==> (&&,<D --> B>,C)>. %1.00;0.90% ... ok
test_second_variable_introduction_induction (test_NAL6.TEST_NAL6)
-'Second variable introduction (induction) ... ok
+'Second variable introduction (induction) ... FAIL
test_symmetry (test_NAL6.TEST_NAL6)
<(*,a,b) --> like>. %1.00;0.90% ... ok
test_uncle (test_NAL6.TEST_NAL6)
@@ -271,12 +232,11 @@
'Second variable introduction (induction) ... ok
test_variable_elimination_deduction_0 (test_NAL6.TEST_NAL6)
<M --> A>. %1.00;0.90% ... ok
-test_variable_elimination_deduction_1 (test_NAL6.TEST_NAL6)
-<M --> A>. %1.00;0.90% ... ok
+test_variables (test_NAL6.TEST_NAL6) ... FAIL
test_abduction (test_NAL7.TEST_NAL7)
'Temporal abudction ... ok
test_abduction_sequence_eliminate_0 (test_NAL7.TEST_NAL7)
-nal7.19.nal ... ok
+nal7.19.nal ... FAIL
test_abduction_sequence_eliminate_1 (test_NAL7.TEST_NAL7)
nal7.19.nal ... FAIL
test_analogy (test_NAL7.TEST_NAL7)
@@ -292,7 +252,7 @@
test_deduction (test_NAL7.TEST_NAL7)
'Temporal deduction ... ok
test_deduction_sequence (test_NAL7.TEST_NAL7)
-nal7.36.nal ... ok
+nal7.36.nal ... FAIL
test_deduction_sequence_eliminate_0 (test_NAL7.TEST_NAL7)
nal7.18.nal ... ERROR
test_deduction_sequence_eliminate_0_1 (test_NAL7.TEST_NAL7) ... ERROR
@@ -313,37 +273,37 @@
test_inference_on_tense_0 (test_NAL7.TEST_NAL7)
'Inference on tense ... FAIL
test_inference_on_tense_1 (test_NAL7.TEST_NAL7)
-' inference on tense ... FAIL
+' inference on tense ... ERROR
test_inference_on_tense_2 (test_NAL7.TEST_NAL7)
' inference on tense ... ok
-test_analogy_0_0__0 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_0_0__0 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
test_analogy_0_0__1 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_analogy_0_0__2 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_analogy_0_0__3 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_0_0__4 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_0_0__4 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
test_analogy_0_0__5 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_0_1__0 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_0_1__0 (test_NAL7.TEST_NAL7_ANALOGY) ... ERROR
test_analogy_0_1__1 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_0_1__2 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_0_1__2 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
test_analogy_0_1__3 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_analogy_0_1__4 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_analogy_0_1__5 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_analogy_1_0__0 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_analogy_1_0__1 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_1_0__2 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_1_0__2 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
test_analogy_1_0__3 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_analogy_1_0__4 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_analogy_1_0__5 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_1_1__0 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
+test_analogy_1_1__0 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_analogy_1_1__1 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_analogy_1_1__2 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_analogy_1_1__3 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
-test_analogy_1_1__4 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
+test_analogy_1_1__4 (test_NAL7.TEST_NAL7_ANALOGY) ... FAIL
test_analogy_1_1__5 (test_NAL7.TEST_NAL7_ANALOGY) ... ok
test_1_0 (test_NAL8.TEST_NAL8)
nal8.1.0.nal ... ok
test_1_1 (test_NAL8.TEST_NAL8)
-nal8.1.1.nal ... FAIL
+nal8.1.1.nal ... ok
test_1_10 (test_NAL8.TEST_NAL8)
nal8.1.10.nal ... ERROR
test_1_11 (test_NAL8.TEST_NAL8)
@@ -353,21 +313,21 @@
test_1_13_var (test_NAL8.TEST_NAL8)
nal8.1.13.nal ... ERROR
test_1_14 (test_NAL8.TEST_NAL8)
-nal8.1.14.nal ... ok
+nal8.1.14.nal ... FAIL
test_1_16 (test_NAL8.TEST_NAL8)
nal8.1.16.nal ... FAIL
test_1_2 (test_NAL8.TEST_NAL8)
nal8.1.2.nal ... FAIL
test_1_3 (test_NAL8.TEST_NAL8)
-nal8.1.3.nal ... ok
+nal8.1.3.nal ... FAIL
test_1_3_var (test_NAL8.TEST_NAL8)
-nal8.1.3.nal ... ERROR
+nal8.1.3.nal ... FAIL
test_1_4 (test_NAL8.TEST_NAL8)
-nal8.1.4.nal ... ok
+nal8.1.4.nal ... FAIL
test_1_4_var (test_NAL8.TEST_NAL8)
nal8.1.4.nal ... FAIL
test_1_5 (test_NAL8.TEST_NAL8)
-nal8.1.5.nal ... ERROR
+nal8.1.5.nal ... ok
test_1_7 (test_NAL8.TEST_NAL8)
nal8.1.7.nal ... FAIL
test_1_7_var (test_NAL8.TEST_NAL8)
@@ -377,7 +337,7 @@
test_1_9 (test_NAL8.TEST_NAL8)
nal8.1.9.nal ... FAIL
test_sequence_0 (test_NAL8.TEST_NAL8)
-(&/, A, B, C). ... ok
+(&/, A, B, C). ... FAIL
test_sequence_1 (test_NAL8.TEST_NAL8)
C! ... FAIL
test_sequence_2 (test_NAL8.TEST_NAL8)
@bowen-xu @ccrock4t So I discovered a problem with swapping out inference engines. I can pass the parameter as we discussed and use either GeneralInference or KanrenEngine and this works. But the problem is that since there are differences in implementation, for example how negation is handled, and some test cases were changed to explicitly use -- instead of frequency of 0. So these test cases will fail if used with GeneralInference engine even thought they work fine on the dev branch. This also applies to some truth values. And anything that relies on truth_analytic being 0.9,0.9 instead of 1.0,1,0.
So I'm looking for your input here. What should we do about it? Technically the GeneralInference still works the same but a lot more test cases fail on this branch compared to the dev branch because the test cases themselves have been changed.
How many cases are failing due to this? At any rate, I guess the new tests / KanrenEngine should take priority. Though, would it be a big or small change to update the GeneralEngine to use negation instead?
How many cases are failing due to this? At any rate, I guess the new tests / KanrenEngine should take priority. Though, would it be a big or small change to update the GeneralEngine to use negation instead?
Due to negation probably just a few. Most are because of the different truth values I believe.
@ccrock4t @bowen-xu Also, here's a fun little comparison of performance between Kanren and General inference engines. https://youtu.be/07NTJhEkiVQ
I guess the takeaway is that there's no easy way to compare the two directly and there is room to improve the computation in KanrenEngine but it is not directly related to miniKanren, which is performant as is, but rather to the way immediate and structural inference are handled.
I understand Inference Engine is quite a hard work... Thank you, Maxim!
ToDo:
variable_elimination
Notes:
test_conversions_between_inheritance_and_similarity