search

haskell / cabal

Official upstream development repository for Cabal and cabal-install
https://haskell.org/cabal
Other
1.61k stars 691 forks source link

Tests UpdateIndexState and ExtraProgPath are weak/fragile #9657

Open Mikolaj opened 8 months ago

Mikolaj commented 8 months ago

While working on https://github.com/haskell/cabal/pull/9609 (now merged), @tomsmeding discovered that the test cabal-testsuite/PackageTests/ExtraProgPath doesn't contain any assertOutputContains, but only requires a failure, which resulted in accepting a failure due to totally unrelated reasons. Also, the test cabal-testsuite/PackageTests/NewUpdate/UpdateIndexState uses port 8000, which sometimes makes it fail, because the port is popular and so often busy. I wonder how the other tests do that --- maybe it makes the most sense to choose a port similar, but different to what other tests choose.

tomsmeding commented 8 months ago

The error for PackageTests/NewUpdate/UpdateIndexState is here: https://github.com/haskell/cabal/actions/runs/7646027766/job/20835397202#step:19:64

CI output included here for posterity and easy access:

CI output 
+ /usr/bin/python3 -m http.server -d /home/runner/work/cabal/cabal/cabal-testsuite/PackageTests/NewUpdate/UpdateIndexState/up[draetter-yi:n0d]e xE-nsctoautnet.edriesdt /Nreetpwoo r-k-.bSioncdk elto.ccaolnhnoesctt :8 0<0s0o
cket: 11>: does not exist (Connection refused). Retrying.
[retry:1] Encountered Network.Socket.connect: : does not exist (Connection refused). Retrying.
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.10/http/server.py", line 1307, in 
    test(
  File "/usr/lib/python3.10/http/server.py", line 1258, in test
    with ServerClass(addr, HandlerClass) as httpd:
  File "/usr/lib/python3.10/socketserver.py", line 453, in __init__
    self.server_activate()
  File "/usr/lib/python3.10/socketserver.py", line 475, in server_activate
    self.socket.listen(self.request_queue_size)
OSError: [Errno 98] Address already in use
[retry:2] Encountered Network.Socket.connect: : does not exist (Connection refused). Retrying.
[retry:3] Encountered Network.Socket.connect: : does not exist (Connection refused). Retrying.
[retry:4] Encountered Network.Socket.connect: : does not exist (Connection refused). Retrying.
[retry:5] Encountered Network.Socket.connect: : does not exist (Connection refused). Retrying.
[retry:6] Encountered Network.Socket.connect: : does not exist (Connection refused). Retrying.
[retry:7] Encountered Network.Socket.connect: : does not exist (Connection refused). Retrying.
[retry:8] Encountered Network.Socket.connect: : does not exist (Connection refused). Retrying.
[retry:9] Encountered Network.Socket.connect: : does not exist (Connection refused). Retrying.
[retry:10] Encountered Network.Socket.connect: : does not exist (Connection refused). Retrying.
The first two lines deinterleave to: 
+ /usr/bin/python3 -m http.server -d /home/runner/work/cabal/cabal/cabal-testsuite/PackageTests/NewUpdate/UpdateIndexState/update-index-state.dist/repo --bind localhost 8000
[retry:0] Encountered Network.Socket.connect: : does not exist (Connection refused). Retrying.