Kernel Crash and errors running Tutorial with custom data

[instabaines]

Description

I am following the 'A first CausalNex tutorial' notebook using a custom dataset JupyterLab. I encountered different issues. I was able to solve some of them.

• Error : ValueError: The given structure is not acyclic. Please review the following cycle: ('A0', 'A1'), ('A1', 'A0')]
• Error: #KeyError: 'A0'
• Kernel crashes on run fit_cpds with train data

  Context

  I am trying to adapt this to my project

  Steps to Reproduce

  data:

  import numpy as np
  import pandas as pd
  np.random.seed(123)
  data=pd.DataFrame({'A'+str(key):np.random.choice([0,1],size=(1000,)) for key in range(181)})
  sm = from_pandas(data,use_gpu=True)

  Processes bn = BayesianNetwork(sm) # this yielded the first error, this was corrected by removing those affected connections bn = bn.fit_cpds(test, method="BayesianEstimator", bayes_prior="K2") # this yileded the second error, it was fixed by running the fit_node_states on the data prior to this step Running fit_cpds again crashed the kernel

Expected Result

Expected a similar output to the tutorial

Actual Result

Canceled future for execute_request message before replies were done The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click here for more info. View Jupyter log for further details. `

Your Environment

• CausalNex version used: 0.11.0
• Python version used: 3.8.13
• Operating system and version: Ubuntu 20.04.4 LTS, running in an Anaconda virtual environment

[ElisabethSesterHussQB]

Hi, thanks for reaching out! We took a look at your issue by running the code you provided. One thing we noticed is that the data you are using might not be best suited when it comes to structure learning as it is very random and therefore the edges all have very small weights. When removing edges using remove_edges_below_threshold we needed to use a threshold around ~0.05 to get rid of enough edges and achieve acyclicity. One step that should be done is also calling get_largest_subgraph as causalnex does not support separated components for now. The expected flow in causalnex would then be the following:

sm = from_pandas()
sm.remove... # remove edges that are wrong by manually removing them or applying a threshold
sm.get_largest_subgraph
# discretise data
discretised_data = discretise data
bn = BayesianNetwork(sm)
bn.fit_node_states(discretised_data)
bn.fit_cpds(discretised_data)

Unfortunately, we weren’t able to recreate your second error. We are happy to take a closer look if you are still facing the same issues. Also any additional code you can share with us would be helpful in finding out the exact cause of the error.