JOSS Review - failing tests

[julienmalard]

I get some failing tests with pdm run pytest on MacOS:

====================================================================== test session starts =======================================================================
platform darwin -- Python 3.10.3, pytest-8.1.1, pluggy-1.4.0
configfile: pyproject.toml
plugins: cov-5.0.0
collected 13 items                                                                                                                                               

test_bivariate_linear.py ...                                                                                                                               [ 23%]
test_gaussian.py .....                                                                                                                                     [ 61%]
test_trees.py .FFFF                                                                                                                                        [100%]

============================================================================ FAILURES ============================================================================
__________________________________________________________________________ test_mixing ___________________________________________________________________________

    def test_mixing():
        x_train = np.loadtxt(
            taweret_wd + 'test/bart_bmm_test_data/2d_x_train.txt').reshape(80, 2)
        x_train = x_train.reshape(2, 80).transpose()

        y_train = np.loadtxt(
            taweret_wd + 'test/bart_bmm_test_data/2d_y_train.txt').reshape(80, 1)

        # Set prior information
        mix.set_prior(
            k=2.5,
            ntree=30,
            overallnu=5,
            overallsd=0.01,
            inform_prior=False)

        # Check tuning & hyper parameters
        assert mix.k == 2.5, "class object k is not set."
        assert mix.ntree == 30, "class object ntree is not set."
        assert mix.overallnu == 5, "class object nu is not set."
        assert mix.overallsd == 0.01, "class object overallsd is not set."
        assert mix.overalllambda == 0.01**2, "class object overalllambda is not set."
        assert mix.inform_prior == False, "class object inform_prior is not set."

        # Train the model
>       fit = mix.train(
            X=x_train,
            y=y_train,
            ndpost=10000,
            nadapt=2000,
            nskip=2000,
            adaptevery=500,
            minnumbot=4)

test_trees.py:73: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../Taweret/mix/trees.py:410: in train
    self._run_model(cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Taweret.mix.trees.Trees object at 0x12b6626e0>, cmd = 'openbtcli'

    def _run_model(self, cmd="openbtcli"):
        """
        Private function, run the cpp program via the command line using
        a subprocess.
        """
        # Check to see if executable is installed via debian
        sh = shutil.which(cmd)

        # Check to see if installed via wheel
        pyinstall = False
        if sh is None:
            pywhl_path = os.popen("pip show openbtmixing").read()
            pywhl_path = pywhl_path.split("Location: ")
            if len(pywhl_path)>1:
                pywhl_path = pywhl_path[1].split("\n")[0] + "/openbtmixing"
                sh = shutil.which(cmd, path=pywhl_path)
                pyinstall = True

        # Execute the subprocess, changing directory when needed
        if sh is None:
            # openbt exe were not found in the current directory -- try the
            # local directory passed in
            sh = shutil.which(cmd, path=self.local_openbt_path)
            if sh is None:
>               raise FileNotFoundError(
                    "Cannot find openbt executables. Please specify the path using the argument local_openbt_path in the constructor.")
E               FileNotFoundError: Cannot find openbt executables. Please specify the path using the argument local_openbt_path in the constructor.

../Taweret/mix/trees.py:857: FileNotFoundError
---------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------
Results stored in temporary path: /var/folders/02/lkk5tj_96tg1q_5p7dlc_1mw0000gn/T/openbtmixing_fz3x2yjx
Running model...
---------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------
WARNING: Package(s) not found: openbtmixing
__________________________________________________________________________ test_predict __________________________________________________________________________

    def test_predict():
        # Get test data
        n_test = 30
        x1_test = np.outer(np.linspace(-3, 3, n_test), np.ones(n_test))
        x2_test = x1_test.copy().transpose()
        f0_test = (np.sin(x1_test) + np.cos(x2_test))
        x_test = np.array([x1_test.reshape(x1_test.size,),
                          x2_test.reshape(x1_test.size,)]).transpose()

        # Read in test results
        pmean_test = np.loadtxt(
            taweret_wd +
            'test/bart_bmm_test_data/2d_pmean.txt')
        eps = 0.10

        # Get predictions
>       ppost, pmean, pci, pstd = mix.predict(X=x_test, ci=0.95)

test_trees.py:107: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../Taweret/mix/trees.py:521: in predict
    self._run_model(cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Taweret.mix.trees.Trees object at 0x12b6626e0>, cmd = 'openbtpred'

    def _run_model(self, cmd="openbtcli"):
        """
        Private function, run the cpp program via the command line using
        a subprocess.
        """
        # Check to see if executable is installed via debian
        sh = shutil.which(cmd)

        # Check to see if installed via wheel
        pyinstall = False
        if sh is None:
            pywhl_path = os.popen("pip show openbtmixing").read()
            pywhl_path = pywhl_path.split("Location: ")
            if len(pywhl_path)>1:
                pywhl_path = pywhl_path[1].split("\n")[0] + "/openbtmixing"
                sh = shutil.which(cmd, path=pywhl_path)
                pyinstall = True

        # Execute the subprocess, changing directory when needed
        if sh is None:
            # openbt exe were not found in the current directory -- try the
            # local directory passed in
            sh = shutil.which(cmd, path=self.local_openbt_path)
            if sh is None:
>               raise FileNotFoundError(
                    "Cannot find openbt executables. Please specify the path using the argument local_openbt_path in the constructor.")
E               FileNotFoundError: Cannot find openbt executables. Please specify the path using the argument local_openbt_path in the constructor.

../Taweret/mix/trees.py:857: FileNotFoundError
---------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------
WARNING: Package(s) not found: openbtmixing
________________________________________________________________________ test_predict_wts ________________________________________________________________________

    def test_predict_wts():
        # Get weights
        n_test = 30
        x1_test = np.outer(np.linspace(-3, 3, n_test), np.ones(n_test))
        x2_test = x1_test.copy().transpose()
        x_test = np.array([x1_test.reshape(x1_test.size,),
                          x2_test.reshape(x1_test.size,)]).transpose()

>       wpost, wmean, wci, wstd = mix.predict_weights(X=x_test, ci=0.95)

test_trees.py:123: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../Taweret/mix/trees.py:604: in predict_weights
    self._run_model(cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Taweret.mix.trees.Trees object at 0x12b6626e0>, cmd = 'openbtmixingwts'

    def _run_model(self, cmd="openbtcli"):
        """
        Private function, run the cpp program via the command line using
        a subprocess.
        """
        # Check to see if executable is installed via debian
        sh = shutil.which(cmd)

        # Check to see if installed via wheel
        pyinstall = False
        if sh is None:
            pywhl_path = os.popen("pip show openbtmixing").read()
            pywhl_path = pywhl_path.split("Location: ")
            if len(pywhl_path)>1:
                pywhl_path = pywhl_path[1].split("\n")[0] + "/openbtmixing"
                sh = shutil.which(cmd, path=pywhl_path)
                pyinstall = True

        # Execute the subprocess, changing directory when needed
        if sh is None:
            # openbt exe were not found in the current directory -- try the
            # local directory passed in
            sh = shutil.which(cmd, path=self.local_openbt_path)
            if sh is None:
>               raise FileNotFoundError(
                    "Cannot find openbt executables. Please specify the path using the argument local_openbt_path in the constructor.")
E               FileNotFoundError: Cannot find openbt executables. Please specify the path using the argument local_openbt_path in the constructor.

../Taweret/mix/trees.py:857: FileNotFoundError
---------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------
WARNING: Package(s) not found: openbtmixing
___________________________________________________________________________ test_sigma ___________________________________________________________________________

    def test_sigma():
        sig_eps = 0.05
        assert np.abs((np.mean(mix.posterior) - 0.1)
>                     ) < sig_eps, "Inaccurate sigma calculation."

test_trees.py:140: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Taweret.mix.trees.Trees object at 0x12b6626e0>

    @property
    def posterior(self):
        '''
        Returns the posterior distribution of the error standard deviation,
        which is learned during the training process.

        Parameters:
        ------------
        :param: None.

        Returns:
        ---------
        :returns: The posterior of the error standard deviation .
        :rtype: np.ndarray

        '''
>       return self._posterior
E       AttributeError: 'Trees' object has no attribute '_posterior'. Did you mean: 'posterior'?

../Taweret/mix/trees.py:180: AttributeError
==================================================================== short test summary info =====================================================================
FAILED test_trees.py::test_mixing - FileNotFoundError: Cannot find openbt executables. Please specify the path using the argument local_openbt_path in the constructor.
FAILED test_trees.py::test_predict - FileNotFoundError: Cannot find openbt executables. Please specify the path using the argument local_openbt_path in the constructor.
FAILED test_trees.py::test_predict_wts - FileNotFoundError: Cannot find openbt executables. Please specify the path using the argument local_openbt_path in the constructor.
FAILED test_trees.py::test_sigma - AttributeError: 'Trees' object has no attribute '_posterior'. Did you mean: 'posterior'?
================================================================== 4 failed, 9 passed in 3.28s ===================================================================

[ominusliticus]

Can you verify if you are using an Apple Silicon chip (M1, M2 or M3)?

[julienmalard]

No, I’m using an older Mac from 2018.

[jcyannotty]

Hi @julienmalard, it looks like there was an issue with the install based on the message "WARNING: Package(s) not found: openbtmixing"

What happens if you run "pip install openbtmixing"? The wheel should download and automatically install the required libraries.

[julienmalard]

In that case, is there any reason not to inclure openbtmixing as a project dependency in the project.toml file?

[ominusliticus]

The openbtmixing is listed in the pyproject.toml dependencies. Maybe a pip install Taweret --upgrade will pull down the missing package, but I just tried a clean install in CodeSpaces and it pulled down the openbtmixing dependency

[julienmalard]

Hello, I tried again - the openbtmixing errors are due to the missing openbtmixing dependency. However, I also get AttributeError: 'Trees' object has no attribute '_posterior'. Did you mean: 'posterior'?. Is this also related to the missing dependency? If so, feel free to close this issue.

[jcyannotty]

Yes - this is exactly related to the missing dependency. The openbtmixing software is used in the Trees class. Despite this, the test_trees.py file is still run with the other pytests. Without the dependency, the model never trains on the test problem, so when you try to access the results stored in "_posterior", there is simply nothing there.

[julienmalard]

Ok, perfect then!