search

biolab / orange3

🍊 :bar_chart: :bulb: Orange: Interactive data analysis
https://orangedatamining.com
Other
4.7k stars 992 forks source link

On MacOS ARM some tests fail because some functions return different results #6820

Closed markotoplak closed 1 week ago

markotoplak commented 4 weeks ago

From https://github.com/biolab/orange3/actions/runs/9317726517/job/25648607002

======================================================================
FAIL: test_max_features_reg (Orange.tests.test_random_forest.RandomForestTest.test_max_features_reg)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/runner/work/orange3/orange3/.tox/orange-released/lib/python3.11/site-packages/Orange/tests/test_random_forest.py", line 134, in test_max_features_reg
    self.assertGreater(diff, 1.2)
AssertionError: 0.030000000000001137 not greater than 1.2

======================================================================
FAIL: test_info (Orange.widgets.evaluate.tests.test_owpermutationplot.TestOWPermutationPlot.test_info)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/runner/work/orange3/orange3/.tox/orange-released/lib/python3.11/site-packages/Orange/widgets/evaluate/tests/test_owpermutationplot.py", line 94, in test_info
    self.assertIn("0.5021", self.widget._info.text())
AssertionError: '0.5021' not found in '\n<table width=100% align="center" style="font-size:11px">\n    <tr style="background:#fefefe">\n        <th style="background:transparent;padding: 2px 4px"></th>\n        <th style="background:transparent;padding: 2px 4px">Corr = 0</th>\n        <th style="background:transparent;padding: 2px 4px">Corr = 100</th>\n    </tr>\n    <tr style="background:#fefefe">\n        <th style="padding: 2px 4px" align=right>Train</th>\n        <td style="padding: 2px 4px" align=right>0.9980</td>\n        <td style="padding: 2px 4px" align=right>0.9996</td>\n    </tr>\n    <tr style="background:#fefefe">\n        <th style="padding: 2px 4px" align=right>CV</th>\n        <td style="padding: 2px 4px" align=right>0.4978</td>\n        <td style="padding: 2px 4px" align=right>0.8951</td>\n    </tr>\n</table>\n            '
markotoplak commented 4 weeks ago

When building installers, I saw that these fails do not happen with

numpy~=1.24.0
scipy~=1.10.0
scikit-learn~=1.3.0

But are problematic with

numpy~=1.25.0
scipy~=1.13.0
scikit-learn~=1.4.0