[Bug] Console not reporting ANSWER even though memory contains correct statement

[maxeeem]

Describe the bug

When trying multistep reasoning example from OpenNARS, the system is able to derive the correct conclusion but it is not reported in Console as ANSWER but as regular OUT.

To Reproduce

Steps to reproduce the behavior:

maxeeem@Maxims-Air PyNARS % python3 -m pynars.Console                                     
PyNARS version v0.0.2 pre-alpha
Setup: rand_seed=137
NARS: Init...
NARS: Run...
NARS: Console.
Input: <a --> b>.
 0.80  0.50  0.95  IN    : <a --> b>.  %1.000;0.900%
Input: <b --> c>.
 0.80  0.50  0.95  IN    : <b --> c>.  %1.000;0.900%
Input: <c --> d>.
 0.80  0.50  0.95  IN    : <c --> d>.  %1.000;0.900%
Input: <a --> d>?
 0.90  0.90  1.00  IN    : <a --> d>?
 0.18  0.50  0.71  OUT   : (--, <b-->c>).  %0.000;0.900%
 0.25  0.50  0.74  OUT   : <c --> b>.  %1.000;0.474%
Input: 2000
                   INFO  : Run 2000 cycles.
 0.13  0.50  0.71  OUT   : (--, <c-->d>).  %0.000;0.900%
 0.17  0.50  0.74  OUT   : <d --> c>.  %1.000;0.474%
 0.80  0.80  0.50  OUT   : <d --> a>?
 0.80  0.80  0.50  OUT   : <a --> c>?

 ...

0.96  0.90  0.63  ANSWER: <a --> d>.  %1.000;0.269%

...

 1.00  0.90  0.86  OUT   : <a --> d>.  %1.000;0.729%

Expected behavior

Higher confidence statement should be the best available answer.

[maxeeem]

@bowen-xu this is related to the conversation here. When writing it as a test case it actually passes since it looks at all of the derived tasks but Console isn't reporting it so somewhere in the control part there is a broken link.

def test_multistep_1(self):
    tasks_derived = process_two_premises(
        '<a --> b>.',
        '<b --> c>.'
    )
    tasks_derived.extend(process_two_premises(
        '<c --> d>.',
        '<a --> d>?',
        200
    ))
    self.assertTrue(
        output_contains(tasks_derived, '<a --> d>. %1.00;0.73%')
    )

[maxeeem]

@bowen-xu Hm... I just tried ConsolePlus and got the right answer....

Which is the right console to use? Console, ConsoleMC or ConsolePlus? Should we continue maintaining all of them or updating the basic one since there's clearly some inconsistency?

EDIT:

Actually not really consistent results for ConsolePlus. Seems to report both and alternate. Something strange with the control mechanism I think.

after running the first 2000 cycles...
.....

Input: <a --> d>?
 0.90  0.90  1.00 IN    :<a --> d>?
Input: 10
            INFO  :Run 10 cycles.
...

 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.86  0.90  0.86 ANSWER:<a --> d>.  %1.000;0.729%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.86  0.90  0.86 ANSWER:<a --> d>.  %1.000;0.729%

[bowen-xu]

@maxeeem Console is the default one. ConsoleMC was implemented by Tangrui@MoonWalker1997 for MultiChannel, and that is outdated. ConsolePlus was implemented by @ARCJ137442 as an optional one.

[bowen-xu]

@bowen-xu Hm... I just tried ConsolePlus and got the right answer....

Which is the right console to use? Console, ConsoleMC or ConsolePlus? Should we continue maintaining all of them or updating the basic one since there's clearly some inconsistency?

EDIT:

Actually not really consistent results for ConsolePlus. Seems to report both and alternate. Something strange with the control mechanism I think.

after running the first 2000 cycles...
.....

Input: <a --> d>?
 0.90  0.90  1.00 IN    :<a --> d>?
Input: 10
            INFO  :Run 10 cycles.
...

 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.86  0.90  0.86 ANSWER:<a --> d>.  %1.000;0.729%
 0.96  0.90  0.64 ANSWER:<a --> d>.  %1.000;0.287%
 0.86  0.90  0.86 ANSWER:<a --> d>.  %1.000;0.729%

@maxeeem I tried the old engine (GeneralEngine), and it output the answer within 500 cycles:

input:

<a --> b>.
<b --> c>.
<c --> d>.
<a --> d>?
500

output:

...
 0.88  0.03  0.09  OUT   : <<$1-->a><=><$1-->d>>. %1.000;0.287%
 0.88  0.03  0.10  OUT   : (&&, <#1-->a>, <#1-->d>). %1.000;0.403%
 0.83  0.28  0.47  OUT   : <c-->c>?
 0.98  0.90  0.86  ANSWER: <a-->d>. %1.000;0.729%
 0.93  0.50  0.65  OUT   : <a-->d>. %1.000;0.297%

[maxeeem]

@maxeeem I tried the old engine (GeneralEngine), and it output the answer within 500 cycles:

Right, very possible. And if you set compositional_enabled to False, then KanrenEngine also produces a result. However, this alone does not explain why the answer isn't produced with that enabled even thought there is a statement with the correct truth value, or why ConsolePlus can give the right answer in some cases but not in others. I still think the control part requires our attention if the goal is to migrate to the new inference engine.

I would recommend reviewing inference_step code and seeing if there is some issue we can identify.

[maxeeem]

@bowen-xu I have another theory. Because of the many more derived statements produced by the new engine, there are many more possible combinations and some relevant items may be pushed out of memory since we default to n_memory = 100 in Console.py run_nars method. If you increase it to 1000 for example and run the inference for 11000 steps like in OpenNARS test case, we get the expected result of 73% confidence.

So to me this once again points to a control thing more than the inference engine i.e. how we pick what to pass to the inference engine at every cycle and how we allocate resources.

[ARCJ137442]

@maxeeem

ConsolePlus was implemented by @ARCJ137442 as an optional one.

As @bowen-xu said, ConsolePlus is an improved version of the default Console written by myself.

You can get more details and enhancements in PR#27.

---

Which is the right console to use? Console, ConsoleMC or ConsolePlus? Should we continue maintaining all of them or updating the basic one since there's clearly some inconsistency?

It's already an issue #35 which is still open yet.

I'm not sure how these 'Console' implementations will be handled, maybe they will eventually be merged into one.

---

Actually not really consistent results for ConsolePlus. Seems to report both and alternate. Something strange with the control mechanism I think.

I checked codes in

https://github.com/bowen-xu/PyNARS/blob/6ee80358254ec14764e51c1da95e20136ec9f731/pynars/Console.py#L160-L166

and

https://github.com/bowen-xu/PyNARS/blob/6ee80358254ec14764e51c1da95e20136ec9f731/pynars/ConsolePlus.py#L520-L550

One of the difference in the reasoning effect between the two consoles lies in the default values of the parameters n_memory and capacity when building the inference engine. The default parameters in Console is smaller than ConsolePlus in n_ roomy and ConsolePlus (100 vs 500)

I also checked codes in

https://github.com/bowen-xu/PyNARS/blob/e5b56a511985607bbb23fc09964e276040b8ade3/pynars/Console.py#L124-L133

and

https://github.com/bowen-xu/PyNARS/blob/e5b56a511985607bbb23fc09964e276040b8ade3/pynars/Interface.py#L414-L422

where the NARSOutput in definited as

https://github.com/bowen-xu/PyNARS/blob/e5b56a511985607bbb23fc09964e276040b8ade3/pynars/Interface.py#L43-L59

and used as

https://github.com/bowen-xu/PyNARS/blob/e5b56a511985607bbb23fc09964e276040b8ade3/pynars/Interface.py#L253-L269

https://github.com/bowen-xu/PyNARS/blob/e5b56a511985607bbb23fc09964e276040b8ade3/pynars/Interface.py#L98-L189

(the code here is a little complicated, and the main purpose is to adapt to some terminals that do not support ANSI escape sequences).

I think there is no functional difference between the two Consoles in printing outputs.