`test/v1x/python/test_condex.py` doesn't finish too often as part of the `Python/MacOS` job

QuTech-Delft / OpenQL

OpenQL: A Portable Quantum Programming Framework for Quantum Accelerators. https://dl.acm.org/doi/10.1145/3474222

https://openql.readthedocs.io

Other

101 stars 45 forks source link

`test/v1x/python/test_condex.py` doesn't finish too often as part of the `Python/MacOS` job #498

Open rturrado opened 1 year ago

rturrado commented 1 year ago

Python/MacOS job is part of the GitHub Actions test workflow.
I've observed from a few weeks ago that test/v1x/python/test_condex.py fails to finish very often.
I wonder if this has something to do with the refactoring I did for OpenQL tests. But I've checked my changes and I haven't seen any significative modification there.
I'm thinking more on the lines that this has to do with some randomness introduced by the OS (and the test feeding itself from that randomness, of course). Something similar happened to us a few weeks ago, where the HiGHS solver was producing two different but perfectly valid solutions depending on the plattorm (x64 and ARM), and our test was expecting only one of them as valid.

jvansomeren commented 1 year ago

You write:

test/v1x/python/test_condex.py fails to finish very often

Can you be more specific:

what is it doing when it doesn't finish, where is it running then?
is it always doing the same thing or more or less at the same position in the test then?

rturrado commented 1 year ago

Many thanks for the comment, Hans.

GitHub Actions don't give much information about what's happening. Since pytest is being run without much verbosity on, you don't even know what test case within test_condex.py is causing the problem. You only know that test_condex.py never finishes. The Test step for the Python/MacOS job simply gets stuck at that point.

Since I don't have a MacOS machine, I will see if I can reproduce this issue by compiling with clang++ on my Linux/x64 machine. If I cannot reproduce it that way, I will add more verbosity to pytest and run this specific job through GitHub Actions to get some more insight.

jvansomeren commented 1 year ago

I can imagine a lot failing, notably in the mapper. The mapper may depend on a random number taken. But we should only start suspecting the mapper when we know that it is consuming its time there. Because Pablo also took care to take the randomness out of the mapper. Looking forward to the results of your bug hunt.

rturrado commented 1 year ago

I could reproduce the issue yesterday in my personal laptop. The test case that is never ending sometimes is the last one, the Toffoli one. I haven't dedicated more time yet to debug it.