Nightly test failure log

jakelishman commented 2 years ago

This is an issue to contain tracking comments from the new nightly CI runs of the full test matrix. Maintainers can subscribe to this issue to get notified whenever the nightly CI encounters issues.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit b2f0964f2c43b7f6104db9e28191210cc6a71bd4. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=35982.

jakelishman commented 2 years ago

Oversubscribed VM, nothing to worry about for us.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit 35fd08b6b764775c600e798670e201597aad7a98. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=36862.

jakelishman commented 2 years ago

Oversubscribed VM again.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit f3877a2983ea78367516cface145d7f8bc555596. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=37349.

jakelishman commented 2 years ago

This failure looks like a potential transient failure in the new quantum Shannon decomposition in #7907. Perhaps the test or the decomposition are slightly buggy.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit f8367e4f43a2faff92d133f859932279b37a10b3. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=37437.

jakelishman commented 2 years ago

Dodgy network connection on a VM - not an actual test failure.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit 91d60bce313b8d041cee6b89505df0a92e377368. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=37679.

jakelishman commented 2 years ago

Oversubscribed VM - nothing to worry about.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit e4e4646f18d6e7d5dae0bc5308bcb5f5c9aa206d. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=37863.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit 6a5adf20e86c74ce507dfb82a4d67ab05522048d. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=38375.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit db612922aa0e0d96d07f40991343da44a31baf1e. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=38387.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit 19fdd32affd37afb58774f40bcf509b16480fe63. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=38419.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit a2780470a22f2050d6bb38316369dd2cd518fdc8. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=38452.

jakelishman commented 2 years ago

The last five failures (while I was away on leave), numbered from oldest to newest:

Some sort of unlucky hang:
```
test.python.quantum_info.states.test_densitymatrix.TestDensityMatrix.test_drawings [] ... inprogress
```
This was only in Windows on Python 3.10, which is also tested in the PR CI and passed. We might want to keep an eye on that, in case it happens again.
Oversubscribed VM.

A stochastic failure on the Windows Python 3.7 job:

test.python.algorithms.test_amplitude_estimators.TestSineIntegral.test_confidence_intervals_2
---------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
   Traceback (most recent call last):

     File "D:\a\1\s\test-job\lib\site-packages\ddt.py", line 191, in wrapper
   return func(self, *args, **kwargs)

     File "D:\a\1\s\test\python\algorithms\test_amplitude_estimators.py", line 412, in test_confidence_intervals
   self.assertGreater(self._statevector.time_taken, 0.0)

     File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\unittest\case.py", line 1251, in assertGreater
   self.fail(self._formatMessage(msg, standardMsg))

     File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\unittest\case.py", line 693, in fail
   raise self.failureException(msg)

   AssertionError: 0.0 not greater than 0.0

Strange build/install failures within matplotlib
Issues caused by setuptools 64.

mtreinish commented 2 years ago

When that happens it means that the test runner worker process stopped for some reason that python doesn't know about so didn't report a result status to stestr. The normal cause of this is a segfault or a similar class of errors. These can be tricky to debug in CI because we typically need extra debug information like kernel logs or a core dump which we don't typically collect.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit d856313aa0739bc30de8e77616bc94e71d651a29. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=38625.

jakelishman commented 2 years ago

This most recent failure is the same as point 3 in my previous comment, and I've seen it in several PR CI runs as well, so we definitely need a fix. Tracked in #8577.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit cb16035a3aaf646f2731fe963275684ca47bc6b4. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=38939.

jakelishman commented 2 years ago

Oversubscribed VM.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit 604cb1ef5ad674fd3873a3f844eb0a526a25a4fc. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=40384.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit 2a8cf79fbee7ab56dc21ebecf613359079f212bd. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=40385.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit 604cb1ef5ad674fd3873a3f844eb0a526a25a4fc. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=40387.

azure-pipelines[bot] commented 2 years ago

Nightly test job failed at commit 2a8cf79fbee7ab56dc21ebecf613359079f212bd. View the logs at https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=40388.