Crash on Cartpole interaction phase

[ntoxeg]

The Cartpole example crashes when trying to run the learning agent (epochs = 5 in the notebook).

ic| msg: 'Learning phase started. (1/5)'
[/usr/local/lib/python3.10/dist-packages/scipy/stats/_continuous_distns.py:608](): RuntimeWarning: divide by zero encountered in _beta_pdf
  return _boost._beta_pdf(x, a, b)
ic| msg: 'Interaction phase started. (1/5)'
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/workspace/01_cartpole.ipynb Cell 21' in <module>
     [17]()[ log_msg(agent_log, f"Interaction phase started. ({i + 1}/{epochs})")
     ]()[18]()[ for j in range(epoch_len):
---> ]()[19]()[     done = agent.control_cycle()
     ]()[20]()[     wrapped_env.render()  # uncomment to see the rendered env
     ]()[21]()[     time.sleep(0.01)

File /workspace/rocca/agents/core.py:1587, in OpencogAgent.control_cycle(self)
   ]()[1584]()[ agent_log.debug("cogscms [count={}]:\n{}".format(len(cogscms), cogscms))
   ]()[1586]()[ # Deduce the action distribution
-> ]()[1587]()[ mxmdl = self.deduce(cogscms)
   ]()[1588]()[ agent_log.debug("mxmdl:\n{}".format(mxmdl_to_str(mxmdl)))
   ]()[1590]()[ # Select the next action

File /workspace/rocca/agents/core.py:1491, in OpencogAgent.deduce(self, cogscms)
   ]()[1456]()[ # For each cognitive schematic estimate the probability of its
   ]()[1457]()[ # context to be true and multiply it by the truth value of the
   ]()[1458]()[ # cognitive schematic, then calculate its weight based on
   (...)
   ]()[1486]()[ # result, as they allegedly exert an unknown influence (via
   ]()[1487]()[ # their invalid parts).
   ]()[1488]()[ ctx_tv = lambda cogscm: get_context_actual_truth(
   ]()[1489]()[     self.atomspace, cogscm, self.cycle_count
   ]()[1490]()[ )
-> ]()[1491]()[ valid_cogscms = [cogscm for cogscm in cogscms if 0.9 < ctx_tv(cogscm).mean]
   ]()[1492]()[ agent_log.fine(
   ]()[1493]()[     "valid_cogscms [count={}]:\n{}".format(len(valid_cogscms), valid_cogscms)
   ]()[1494]()[ )
   ]()[1496]()[ # Size of the complete data set, including all observations
   ]()[1497]()[ # used to build the models.
   ]()[1498]()[ #
   ]()[1499]()[ # Needs to be set before calling self.weight

File /workspace/rocca/agents/core.py:1491, in <listcomp>(.0)
   ]()[1456]()[ # For each cognitive schematic estimate the probability of its
   ]()[1457]()[ # context to be true and multiply it by the truth value of the
   ]()[1458]()[ # cognitive schematic, then calculate its weight based on
   (...)
   ]()[1486]()[ # result, as they allegedly exert an unknown influence (via
   ]()[1487]()[ # their invalid parts).
   ]()[1488]()[ ctx_tv = lambda cogscm: get_context_actual_truth(
   ]()[1489]()[     self.atomspace, cogscm, self.cycle_count
   ]()[1490]()[ )
-> ]()[1491]()[ valid_cogscms = [cogscm for cogscm in cogscms if 0.9 < ctx_tv(cogscm).mean]
   ]()[1492]()[ agent_log.fine(
   ]()[1493]()[     "valid_cogscms [count={}]:\n{}".format(len(valid_cogscms), valid_cogscms)
   ]()[1494]()[ )
   ]()[1496]()[ # Size of the complete data set, including all observations
   ]()[1497]()[ # used to build the models.
   ]()[1498]()[ #
   ]()[1499]()[ # Needs to be set before calling self.weight

File /workspace/rocca/agents/core.py:1488, in OpencogAgent.deduce.<locals>.<lambda>(cogscm)
   ]()[1454]()[ agent_log.fine("deduce(cogscms={})".format(cogscms))
   ]()[1456]()[ # For each cognitive schematic estimate the probability of its
   ]()[1457]()[ # context to be true and multiply it by the truth value of the
   ]()[1458]()[ # cognitive schematic, then calculate its weight based on
   (...)
   ]()[1486]()[ # result, as they allegedly exert an unknown influence (via
   ]()[1487]()[ # their invalid parts).
-> ]()[1488]()[ ctx_tv = lambda cogscm: get_context_actual_truth(
   ]()[1489]()[     self.atomspace, cogscm, self.cycle_count
   ]()[1490]()[ )
   ]()[1491]()[ valid_cogscms = [cogscm for cogscm in cogscms if 0.9 < ctx_tv(cogscm).mean]
   ]()[1492]()[ agent_log.fine(
   ]()[1493]()[     "valid_cogscms [count={}]:\n{}".format(len(valid_cogscms), valid_cogscms)
   ]()[1494]()[ )

File /workspace/rocca/agents/utils.py:832, in get_context_actual_truth(atomspace, cogscm, i)
    ]()[825]()[ body = AndLink(
    ]()[826]()[     PresentLink(*stamped_present_clauses),
    ]()[827]()[     IsClosedLink(*stamped_present_clauses),
    ]()[828]()[     IsTrueLink(*stamped_present_clauses),
    ]()[829]()[     *virtual_clauses
    ]()[830]()[ )
    ]()[831]()[ query = SatisfactionLink(vardecl, body)
--> ]()[832]()[ tv = execute_atom(atomspace, query)
    ]()[833]()[ return tv

File execute.pyx:14, in opencog.execute.execute_atom()

RuntimeError: BUG! Still have ungrounded clauses!! (/home/opencog/atomspace/opencog/query/NextSearchMixin.cc:154)]()

[ntoxeg]

Apparently executing such a query can crash:

(SatisfactionLink
  (VariableSet
    (VariableNode "$Y") ; [49d39f8897aa102b][7]
  ) ; [daa7d3e50925b2a8][7]
  (AndLink
    (IsTrueLink
      (AtTimeLink (stv 1 1)
        (EvaluationLink
          (PredicateNode "Reward") ; [155bb4d713db0d51][7]
          (NumberNode "1") ; [2cf0956d543cff8e][7]
        ) ; [bd5beca0b979487b][7]
        (SLink
          (SLink
            (SLink
              (SLink
        ) ; [decddbebf3f8f28d][7]
      ) ; [94bc9bf5af0c4715][7]
    ) ; [d874ba32db613480][7]
  ) ; [88ed8b9be882c38e][7]
) ; [898f9dbc6c32c8fa][7]

[ntoxeg]

I’m guessing it’s because there is the $Y variable that doesn’t appear in the body, hence it can’t be grounded? @ngeiswei

[ngeiswei]

@ntoxeg, could you make sure you are using revision https://github.com/opencog/atomspace/commit/b158262242b00a4939cd1451c17b6ce99dda8cc9 of the atomspace repo?

[ntoxeg]

@ngeiswei I can but this is still a bug. Is this reported for AtomSpace? I can open an issue if you want. If this is strictly an AtomSpace bug, then I will close this of course.

[ngeiswei]

@ntoxeg, looking closer, it's probably a problem with ROCCA, though it would still be good if you check with the suggested SHA, just in case.

[ntoxeg]

@ngeiswei I’ve tested with that AtomSpace commit and… got a segmentation fault (on the learning agent). The Chase example gets: /usr/local/lib/opencog/libure.so: undefined symbol: _ZN7opencog9AtomSpace12extract_atomERKNS_6HandleEb. That means you can’t just choose an arbitrary AtomSpace version — there are other things that change around (I’m guessing mainly URE). So unless you will figure out what URE commit I should also use, then I don’t think we’ll get much out of this lead.

[ngeiswei]

@ntoxeg, you probably just need to recompile/reinstall the URE as well, only the atomspace requires a special SHA, the other components require the latest revisions.

[ntoxeg]

@ngeiswei nope, reinstalling latest URE just makes both examples crash with segmentation fault now.

[ntoxeg]

@ngeiswei after reinstalling both Miner and PLN at least Chase seems to be running, have to check the rest.

[ntoxeg]

@ngeiswei CartPole crashes with the same message. The query this time is larger though:

(SatisfactionLink
  (VariableSet
    (VariableNode "$X") ; [5e83f0863b0b7bf4][7]
  ) ; [98fbb2acfc0ccdb8][7]
  (AndLink
    (IsClosedLink
      (AtTimeLink
        (EvaluationLink
          (PredicateNode "Reward") ; [155bb4d713db0d51][7]
          (NumberNode "1") ; [2cf0956d543cff8e][7]
        ) ; [d3cee8bdda06ffcb][7]
        (SLink
          (SLink
            (SLink
              (SLink
                (SLink
                  (SLink
                    (SLink
                      (SLink
                        (SLink
                          (SLink
                            (SLink
                              (SLink
                                (SLink
                                  (SLink
                                    (ZLink
                                    ) ; [800fbffffffe8ce4][7]
                                  ) ; [da5f815ba9d4009f][7]
                                ) ; [f5363085cdea2ffe][7]
                              ) ; [fd6dd72b31edbe31][7]
                            ) ; [f28a7ade63d17cf3][7]
                          ) ; [fc2d06252a1a9d47][7]
                        ) ; [e3c64b619861796f][7]
                      ) ; [a1af099fa1fc2a5c][7]
                    ) ; [bf3bc83cdf974bac][7]
                  ) ; [9c3cf1b80bffa691][7]
                ) ; [b1dbf2895c0ac39a][7]
              ) ; [99898d838458a822][7]
            ) ; [a88f491105b98a22][7]
          ) ; [f296d319dc6358ba][7]
        ) ; [daa9ba861899d8d9][7]
      ) ; [9f08e4fd44bc31f6][7]
      (AtTimeLink
        (EvaluationLink
          (PredicateNode "Pole Angular Velocity") ; [7b3b72b6dcbe731b][7]
          (VariableNode "$X") ; [5e83f0863b0b7bf4][7]
        ) ; [cd316eb8e1482476][7]
        (SLink
          (SLink
            (SLink
              (SLink
                (SLink
                  (SLink
                    (SLink
                      (SLink
                        (SLink
                          (SLink
                            (SLink
                              (SLink
                                (SLink
                                  (SLink
                                    (ZLink
                                    ) ; [800fbffffffe8ce4][7]
                                  ) ; [da5f815ba9d4009f][7]
                                ) ; [f5363085cdea2ffe][7]
                              ) ; [fd6dd72b31edbe31][7]
                            ) ; [f28a7ade63d17cf3][7]
                          ) ; [fc2d06252a1a9d47][7]
                        ) ; [e3c64b619861796f][7]
                      ) ; [a1af099fa1fc2a5c][7]
                    ) ; [bf3bc83cdf974bac][7]
                  ) ; [9c3cf1b80bffa691][7]
                ) ; [b1dbf2895c0ac39a][7]
              ) ; [99898d838458a822][7]
            ) ; [a88f491105b98a22][7]
          ) ; [f296d319dc6358ba][7]
        ) ; [daa9ba861899d8d9][7]
      ) ; [ee9692c1461d1cda][7]
    ) ; [87544c18e1942c7b][7]
    (IsTrueLink
      (AtTimeLink
        (EvaluationLink
          (PredicateNode "Reward") ; [155bb4d713db0d51][7]
          (NumberNode "1") ; [2cf0956d543cff8e][7]
        ) ; [d3cee8bdda06ffcb][7]
        (SLink
          (SLink
            (SLink
              (SLink
                (SLink
                  (SLink
                    (SLink
                      (SLink
                        (SLink
                          (SLink
                            (SLink
                              (SLink
                                (SLink
                                  (SLink
                                    (ZLink
                                    ) ; [800fbffffffe8ce4][7]
                                  ) ; [da5f815ba9d4009f][7]
                                ) ; [f5363085cdea2ffe][7]
                              ) ; [fd6dd72b31edbe31][7]
                            ) ; [f28a7ade63d17cf3][7]
                          ) ; [fc2d06252a1a9d47][7]
                        ) ; [e3c64b619861796f][7]
                      ) ; [a1af099fa1fc2a5c][7]
                    ) ; [bf3bc83cdf974bac][7]
                  ) ; [9c3cf1b80bffa691][7]
                ) ; [b1dbf2895c0ac39a][7]
              ) ; [99898d838458a822][7]
            ) ; [a88f491105b98a22][7]
          ) ; [f296d319dc6358ba][7]
        ) ; [daa9ba861899d8d9][7]
      ) ; [9f08e4fd44bc31f6][7]
      (AtTimeLink
        (EvaluationLink
          (PredicateNode "Pole Angular Velocity") ; [7b3b72b6dcbe731b][7]
          (VariableNode "$X") ; [5e83f0863b0b7bf4][7]
        ) ; [cd316eb8e1482476][7]
        (SLink
          (SLink
            (SLink
              (SLink
                (SLink
                  (SLink
                    (SLink
                      (SLink
                        (SLink
                          (SLink
                            (SLink
                              (SLink
                                (SLink
                                  (SLink
                                    (ZLink
                                    ) ; [800fbffffffe8ce4][7]
                                  ) ; [da5f815ba9d4009f][7]
                                ) ; [f5363085cdea2ffe][7]
                              ) ; [fd6dd72b31edbe31][7]
                            ) ; [f28a7ade63d17cf3][7]
                          ) ; [fc2d06252a1a9d47][7]
                        ) ; [e3c64b619861796f][7]
                      ) ; [a1af099fa1fc2a5c][7]
                    ) ; [bf3bc83cdf974bac][7]
                  ) ; [9c3cf1b80bffa691][7]
                ) ; [b1dbf2895c0ac39a][7]
              ) ; [99898d838458a822][7]
            ) ; [a88f491105b98a22][7]
          ) ; [f296d319dc6358ba][7]
        ) ; [daa9ba861899d8d9][7]
      ) ; [ee9692c1461d1cda][7]
    ) ; [bb0076f4b622c9e4][7]
    (PresentLink
      (AtTimeLink
        (EvaluationLink
          (PredicateNode "Reward") ; [155bb4d713db0d51][7]
          (NumberNode "1") ; [2cf0956d543cff8e][7]
        ) ; [d3cee8bdda06ffcb][7]
        (SLink
          (SLink
            (SLink
              (SLink
                (SLink
                  (SLink
                    (SLink
                      (SLink
                        (SLink
                          (SLink
                            (SLink
                              (SLink
                                (SLink
                                  (SLink
                                    (ZLink
                                    ) ; [800fbffffffe8ce4][7]
                                  ) ; [da5f815ba9d4009f][7]
                                ) ; [f5363085cdea2ffe][7]
                              ) ; [fd6dd72b31edbe31][7]
                            ) ; [f28a7ade63d17cf3][7]
                          ) ; [fc2d06252a1a9d47][7]
                        ) ; [e3c64b619861796f][7]
                      ) ; [a1af099fa1fc2a5c][7]
                    ) ; [bf3bc83cdf974bac][7]
                  ) ; [9c3cf1b80bffa691][7]
                ) ; [b1dbf2895c0ac39a][7]
              ) ; [99898d838458a822][7]
            ) ; [a88f491105b98a22][7]
          ) ; [f296d319dc6358ba][7]
        ) ; [daa9ba861899d8d9][7]
      ) ; [9f08e4fd44bc31f6][7]
      (AtTimeLink
        (EvaluationLink
          (PredicateNode "Pole Angular Velocity") ; [7b3b72b6dcbe731b][7]
          (VariableNode "$X") ; [5e83f0863b0b7bf4][7]
        ) ; [cd316eb8e1482476][7]
        (SLink
          (SLink
            (SLink
              (SLink
                (SLink
                  (SLink
                    (SLink
                      (SLink
                        (SLink
                          (SLink
                            (SLink
                              (SLink
                                (SLink
                                  (SLink
                                    (ZLink
                                    ) ; [800fbffffffe8ce4][7]
                                  ) ; [da5f815ba9d4009f][7]
                                ) ; [f5363085cdea2ffe][7]
                              ) ; [fd6dd72b31edbe31][7]
                            ) ; [f28a7ade63d17cf3][7]
                          ) ; [fc2d06252a1a9d47][7]
                        ) ; [e3c64b619861796f][7]
                      ) ; [a1af099fa1fc2a5c][7]
                    ) ; [bf3bc83cdf974bac][7]
                  ) ; [9c3cf1b80bffa691][7]
                ) ; [b1dbf2895c0ac39a][7]
              ) ; [99898d838458a822][7]
            ) ; [a88f491105b98a22][7]
          ) ; [f296d319dc6358ba][7]
        ) ; [daa9ba861899d8d9][7]
      ) ; [ee9692c1461d1cda][7]
    ) ; [df51cbdf6cac8b61][7]
  ) ; [c9b539764523b0bd][7]
) ; [c90c100d507c9080][7]

Chase works again.

[ngeiswei]

OK, I'll investigate when I get a chance. Thanks for the report.

[ngeiswei]

@ntoxeg, I cannot reproduce that bug, neither with 01_cartpole.ipynb nor with examples/cartpole.py, even after multiple attempts. Does the bug occur systematically on your system? With examples/cartpole.py as well?

[ntoxeg]

@ngeiswei examples/cartpole.py doesn’t count because it doesn’t use the learning agent (it uses just the FixedCartPoleAgent). In the notebook in the last cell you have to change the epochs variable to something greater than 1. Unfortunately you have to wait like 15 minutes for learning to see that.

I am now running LearningCartPoleAgent again just to make sure if it happens again (probably yes).

[ntoxeg]

I can confirm that the crash still happens.

[ngeiswei]

I can reproduce the crash, will look deeper into it.

[ngeiswei]

The culprit query is here (ending with the txt suffix to please github). It seems like a pattern matcher bug to me. To further investigate we'd need to create a standalone test case. I'll see if I can do that next.

[ngeiswei]

I can confirm that it is a pattern matcher bug (or at least an atomspace bug of some sort). The following archive contains a standalone atomspace and query to reproduce the problem. To create an actual issue on the atomspace repo would require first to reduce it to the smallest possible size, cause both the query and the atomspace being queried are gigantic. issue-59-test.tar.gz