test_signal fails in AMD64 Debian PGO 3.x

pablogsal commented 5 years ago

BPO	36013
Nosy	@gpshead, @pablogsal
PRs	python/cpython#11902
Files	19592

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/gpshead' closed_at = created_at = labels = ['3.8', 'type-bug', 'tests'] title = 'test_signal fails in AMD64 Debian PGO 3.x' updated_at = user = 'https://github.com/pablogsal' ``` bugs.python.org fields: ```python activity = actor = 'gregory.p.smith' assignee = 'gregory.p.smith' closed = True closed_date = closer = 'gregory.p.smith' components = ['Tests'] creation = creator = 'pablogsal' dependencies = [] files = ['48145'] hgrepos = [] issue_num = 36013 keywords = ['patch'] message_count = 7.0 messages = ['335729', '335732', '335735', '335736', '335737', '335739', '335741'] nosy_count = 2.0 nosy_names = ['gregory.p.smith', 'pablogsal'] pr_nums = ['11902'] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue36013' versions = ['Python 3.8'] ```

pablogsal commented 5 years ago

test_signal fails in AMD64 Debian PGO 3.x:

== Tests result: FAILURE then FAILURE == 406 tests OK. 10 slowest tests:

test_concurrent_futures: 2 min 37 sec
test_multiprocessing_spawn: 2 min 3 sec
test_multiprocessing_forkserver: 1 min 28 sec
test_multiprocessing_fork: 1 min 17 sec
test_gdb: 46 sec 874 ms
test_asyncio: 46 sec 166 ms
test_tools: 37 sec 582 ms
test_io: 35 sec 518 ms
test_tokenize: 34 sec 631 ms
test_subprocess: 34 sec 302 ms 1 test failed: test_signal 13 tests skipped: test_devpoll test_ioctl test_kqueue test_msilib test_ossaudiodev test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio test_winreg test_winsound test_zipfile64

https://buildbot.python.org/all/#/builders/47/builds/2251/steps/4/logs/stdio

test_valid_signals (test.test_signal.WindowsSignalTests) ... skipped 'Windows specific' \====================================================================== FAIL: test_keyboard_interrupt_communicated_to_shell (test.test_signal.PosixTests) KeyboardInterrupt exits such that shells detect a ^C. ----------------------------------------------------------------------

Traceback (most recent call last):
  File "/var/lib/buildbot/slaves/enable-optimizations-bot/3.x.gps-debian-profile-opt.nondebug/build/Lib/test/test_signal.py", line 121, in test_keyboard_interrupt_communicated_to_shell
    self.assertNotIn(b"TESTFAIL", process.stdout)
AssertionError: b'TESTFAIL' unexpectedly found in b'TESTFAIL using bash 4.4.12(1)-release\n'

This may be related to PR11862

gpshead commented 5 years ago

This means either that the bash minimum version needs upping to a patch level or that the test is actually flaky and both I and all of the CI got lucky when running it (i'm not actually sure if any of the CI actually ran this test or just skipped it - the test.regrtest output on CI is never verbose so I can't tell).

if i can't untangle bash versions, the other alternative is to just not bother with a shell based test for this as it is too complicated due to depending on exact behaviors of a shell testing an interactive feature in fake interactive mode.

changing the child process to just "raise KeyboardInterrupt" as is done in the equivalent Windows test is sufficient.

pablogsal commented 5 years ago

I scheduled a rebuild to check if the failure is consistent.

pablogsal commented 5 years ago

I have tested in Ubuntu with the same version of bash 4.4.12(1)-release and the test succeeds:

root@d7d8cf5d89bf:/src# echo $BASH_VERSION 4.4.12(1)-release

root@d7d8cf5d89bf:/src# ./python -m test test_signal -m test_keyboard_interrupt_communicated_to_shell Run tests sequentially 0:00:00 load avg: 0.77 [1/1] test_signal b'4.4.12(1)-release\n' [4, 4]

== Tests result: SUCCESS ==

1 test OK.

Total duration: 74 ms Tests result: SUCCESS

gpshead commented 5 years ago

testing on my end with various bash versions shows that bash 4.3.0 and 4.4.0 and 5.0.0 all work fine on my linux and macos systems. caveat, i'm launching regrtest from a login shell (in case that influences child bash processes behavior).

i'm going to abandon bash for this test.

pablogsal commented 5 years ago

Could it be a race condition the signal delivery?

Technically, the process sends itself a signal and never waits for it, it just exits. The signal can arrive after the process has exited normally. Or I am missing something obvious in this reasoning?

gpshead commented 5 years ago

New changeset 414c625a7ea58943f0b1bc79d095d667d78db013 by Gregory P. Smith in branch 'master': bpo-36013: delete fragile interactive shell SIGINT test (GH-11902) https://github.com/python/cpython/commit/414c625a7ea58943f0b1bc79d095d667d78db013

python / cpython

test_signal fails in AMD64 Debian PGO 3.x #80194