glatterf42
commented
6 months ago Possibly related to #467.
Closed glatterf42 closed 3 weeks ago
glatterf42
commented
6 months ago Possibly related to #467.
glatterf42
commented
6 months ago Here's a list of flaky tests I recorded since 11.01.2024:
[ ] test_py_transport
FAILED ixmp/tests/test_tutorials.py::test_py_transport - nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
------------------
# launch the ix modeling platform using the default back end
mp = ixmp.Platform()
# The following lines have the same effect:
# mp = ixmp.Platform(name='local') # Default local database
# mp = ixmp.Platform(name='default') # Default database -> 'local'
------------------
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
File Platform.java:110, in at.ac.iiasa.ixmp.Platform.<init>()
File Platform.java:163, in at.ac.iiasa.ixmp.Platform.initPlatform()
File DbDAO.java:238, in at.ac.iiasa.ixmp.database.DbDAO.create()
File DbDAO.java:318, in at.ac.iiasa.ixmp.database.DbDAO.updateSchemaIfNecessary()
File Flyway.java:910, in org.flywaydb.core.Flyway.migrate()
File Flyway.java:1238, in org.flywaydb.core.Flyway.execute()
File Flyway.java:910, in org.flywaydb.core.Flyway$1.execute()
File Flyway.java:918, in org.flywaydb.core.Flyway$1.execute()
File Flyway.java:78, in org.flywaydb.core.Flyway.access$100()
File Flyway.java:1037, in org.flywaydb.core.Flyway.doValidate()
Exception: Java Exception
The above exception was the direct cause of the following exception:
org.flywaydb.core.api.FlywayException Traceback (most recent call last)
File ~/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/ixmp/backend/jdbc.py:297, in JDBCBackend.__init__(self, jvmargs, **kwargs)
296 try:
--> 297 self.jobj = java.Platform("Python", properties)
298 except java.NoClassDefFoundError as e: # pragma: no cover
org.flywaydb.core.api.FlywayException: org.flywaydb.core.api.FlywayException: Validate failed: Detected failed migration to version 1 (hsql base version)
During handling of the above exception, another exception occurred:
RuntimeError Traceback (most recent call last)
Cell In[2], line 2
1 # launch the ix modeling platform using the default back end
----> 2 mp = ixmp.Platform()
4 # The following lines have the same effect:
5 # mp = ixmp.Platform(name='local') # Default local database
6 # mp = ixmp.Platform(name='default') # Default database -> 'local'
File ~/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/ixmp/core/platform.py:94, in Platform.__init__(self, name, backend, **backend_args)
88 raise ValueError(
89 f"backend class {repr(backend_class_name)} not among "
90 + str(sorted(BACKENDS.keys()))
91 )
93 # Instantiate the backend
---> 94 self._backend = backend_class(**kwargs)
File ~/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/ixmp/backend/jdbc.py:320, in JDBCBackend.__init__(self, jvmargs, **kwargs)
318 else:
319 _raise_jexception(e)
--> 320 raise RuntimeError(f"{msg}\n(Java: {jclass})")
322 # Set the log level
323 self.set_log_level(log_level)
RuntimeError: when initializing database:
(Java: org.flywaydb.core.api.FlywayException)
On: macos-latest-py3.8, macos-latest-py3.10[ ] test_py_transport_scenario
FAILED ixmp/tests/test_tutorials.py::test_py_transport_scenario - nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
------------------
# launch the ix modeling platform using the default local database
mp = ixmp.Platform()
------------------
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
File Platform.java:110, in at.ac.iiasa.ixmp.Platform.<init>()
File Platform.java:163, in at.ac.iiasa.ixmp.Platform.initPlatform()
File DbDAO.java:238, in at.ac.iiasa.ixmp.database.DbDAO.create()
File DbDAO.java:318, in at.ac.iiasa.ixmp.database.DbDAO.updateSchemaIfNecessary()
File Flyway.java:910, in org.flywaydb.core.Flyway.migrate()
File Flyway.java:1238, in org.flywaydb.core.Flyway.execute()
File Flyway.java:910, in org.flywaydb.core.Flyway$1.execute()
File Flyway.java:918, in org.flywaydb.core.Flyway$1.execute()
File Flyway.java:78, in org.flywaydb.core.Flyway.access$100()
File Flyway.java:1037, in org.flywaydb.core.Flyway.doValidate()
Exception: Java Exception
The above exception was the direct cause of the following exception:
org.flywaydb.core.api.FlywayException Traceback (most recent call last)
File ~/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/ixmp/backend/jdbc.py:297, in JDBCBackend.__init__(self, jvmargs, **kwargs)
296 try:
--> 297 self.jobj = java.Platform("Python", properties)
298 except java.NoClassDefFoundError as e: # pragma: no cover
org.flywaydb.core.api.FlywayException: org.flywaydb.core.api.FlywayException: Validate failed: Detected failed migration to version 1 (hsql base version)
During handling of the above exception, another exception occurred:
RuntimeError Traceback (most recent call last)
Cell In[2], line 2
1 # launch the ix modeling platform using the default local database
----> 2 mp = ixmp.Platform()
File ~/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/ixmp/core/platform.py:94, in Platform.__init__(self, name, backend, **backend_args)
88 raise ValueError(
89 f"backend class {repr(backend_class_name)} not among "
90 + str(sorted(BACKENDS.keys()))
91 )
93 # Instantiate the backend
---> 94 self._backend = backend_class(**kwargs)
File ~/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/ixmp/backend/jdbc.py:320, in JDBCBackend.__init__(self, jvmargs, **kwargs)
318 else:
319 _raise_jexception(e)
--> 320 raise RuntimeError(f"{msg}\n(Java: {jclass})")
322 # Set the log level
323 self.set_log_level(log_level)
RuntimeError: when initializing database:
(Java: org.flywaydb.core.api.FlywayException)
On: macos-latest-py3.8, macos-latest-py3.10[ ] test_R_transport
FAILED ixmp/tests/test_tutorials.py::test_R_transport - nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
------------------
# launch the ix modeling platform using a local HSQL database instance
mp <- ixmp$Platform()
------------------
when initializing database:
(Java: org.flywaydb.core.api.FlywayException)Traceback:
1. ixmp$Platform()
2. py_call_impl(callable, call_args$unnamed, call_args$named)
On: macos-latest-py3.8, macos-latest-py3.10[ ] test_R_transport_scenario
FAILED ixmp/tests/test_tutorials.py::test_R_transport_scenario - nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
------------------
# launch the ix modeling platform using the local default database
mp <- ixmp$Platform()
------------------
when initializing database:
(Java: org.flywaydb.core.api.FlywayException)Traceback:
1. ixmp$Platform()
2. py_call_impl(callable, call_args$unnamed, call_args$named)
On: macos-latest-py3.8, macos-latest-py3.10[x] test_del_ts
FAILED ixmp/tests/backend/test_jdbc.py::test_del_ts - assert 8 == (1 + 8)
+ where 8 = len(<WeakKeyDictionary at 0x120d52330>)
+ where <WeakKeyDictionary at 0x120d52330> = <ixmp.backend.jdbc.JDBCBackend object at 0x14b0e3890>.jindex
+ where <ixmp.backend.jdbc.JDBCBackend object at 0x14b0e3890> = <ixmp.core.platform.Platform object at 0x14b0e2bd0>._backend
On: macos-latest-py3.12[x] test_last_update[TimeSeries]
FAILED ixmp/tests/core/test_timeseries.py::TestTimeSeries::test_last_update[TimeSeries] - assert datetime.timedelta(seconds=1, microseconds=390684) < datetime.timedelta(seconds=1)
+ where datetime.timedelta(seconds=1, microseconds=390684) = abs((datetime.datetime(2024, 1, 12, 6, 36, 10, 712000) - datetime.datetime(2024, 1, 12, 6, 36, 12, 102684)))
+ where datetime.datetime(2024, 1, 12, 6, 36, 12, 102684) = <built-in method now of type object at 0x109bc0ba8>()
+ where <built-in method now of type object at 0x109bc0ba8> = datetime.now
+ and datetime.timedelta(seconds=1) = timedelta(seconds=1)
On: macos-latest-py3.12[ ] test_export_timeseries_data
FAILED ixmp/tests/core/test_platform.py::test_export_timeseries_data - AssertionError: DataFrame are different
DataFrame shape mismatch
[left]: (2, 10)
[right]: (0, 10)
On: macos-latest-py3.8, macos-latest-py3.12[x] test_connect_message
FAILED ixmp/tests/backend/test_jdbc.py::test_connect_message - assert "connected to database 'jdbc:hsqldb:mem://ixmptest' (user: ixmp)..." in ''
+ where '' = CaptureResult(out='', err='').out
On: windows-latest-py3.9[x] test_del_ts
FAILED ixmp/tests/backend/test_jdbc.py::test_del_ts - assert 8 == (1 + 8)
+ where 8 = len(<WeakKeyDictionary at 0x1264df1a0>)
+ where <WeakKeyDictionary at 0x1264df1a0> = <ixmp.backend.jdbc.JDBCBackend object at 0x151195730>.jindex
+ where <ixmp.backend.jdbc.JDBCBackend object at 0x151195730> = <ixmp.core.platform.Platform object at 0x1511949e0>._backend
On: macos-latest-py3.12[x] test_close
FAILED ixmp/tests/backend/test_jdbc.py::test_close - AssertionError: assert 'Database connection could not be closed or was already closed' in ''
+ where '' = CaptureResult(out='', err='').out
On: windows-latest-py3.8[x] test_del_ts
FAILED ixmp/tests/backend/test_jdbc.py::test_del_ts - AssertionError: ("<frame at 0x146cd3510, file '/Users/runner/work/ixmp/ixmp/ixmp/backend/jdbc.py', line 732, code check_out>
<frame a..._>
<frame at 0x1252f9e40, file '/Users/runner/work/ixmp/ixmp/ixmp/testing/data.py', line 220, code make_dantzig>", [])
assert 1 == (20 - 1)
+ where 20 = getrefcount(<ixmp.core.scenario.Scenario object at 0x146b920f0>)
On: macos-latest-py3.12[x] test_platform_copy
FAILED ixmp/tests/test_cli.py::test_platform_copy - AssertionError: Copy /tmp/pytest-of-runner/pytest-0/popen-gw0/test_platform_copy0/p2.script.new → /tmp/pytest-of-runner/pytest-0/popen-gw0/test_platform_copy0/p3.new
Copy /tmp/pytest-of-runner/pytest-0/popen-gw0/test_platform_copy0/p2.properties.new → /tmp/pytest-of-runner/pytest-0/popen-gw0/test_platform_copy0/p3.new
assert (not True or 1 == 0)
+ where 1 = <Result FileNotFoundError(2, 'No such file or directory')>.exit_code
On: ubuntu-latest-py3.12, macos-latest-py3.12
As mentioned in https://github.com/iiasa/ixmp/pull/510, pytest-xdist allows us to run tests on multiple tasks simultaneously, decreasing our run time. However, some tests show flaky behaviour since they attempt to read/write from processes they don't belong to. https://github.com/iiasa/message_ix/pull/765#issuecomment-1851995857 mentions a method with which to resolve this flakiness and successfully implements it on several tests in that repo. We should add the same/similar fixes here and then revert the mitigation added in https://github.com/iiasa/ixmp/pull/510.