Crash while compiling Python 3000 in OpenBSD 4.4

[fef5b636-c1d6-46b0-8843-2d43506ab7d2]

BPO	3685
Nosy	@loewis, @warsaw, @hyeshik
Files	fix_wcschr_openbsd.diff fix_wcschr_generic.patch wcslen.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['interpreter-core', 'type-crash', 'release-blocker'] title = 'Crash while compiling Python 3000 in OpenBSD 4.4' updated_at = user = 'https://bugs.python.org/henryprecheur' ``` bugs.python.org fields: ```python activity = actor = 'barry' assignee = 'none' closed = True closed_date = closer = 'barry' components = ['Interpreter Core'] creation = creator = 'henry.precheur' dependencies = [] files = ['11265', '11369', '11537'] hgrepos = [] issue_num = 3685 keywords = ['patch', 'needs review'] message_count = 9.0 messages = ['71968', '71969', '71977', '72008', '72009', '72472', '73498', '73941', '74886'] nosy_count = 4.0 nosy_names = ['loewis', 'barry', 'hyeshik.chang', 'henry.precheur'] pr_nums = [] priority = 'release blocker' resolution = 'accepted' stage = None status = 'closed' superseder = None type = 'crash' url = 'https://bugs.python.org/issue3685' versions = ['Python 3.0'] ```

[fef5b636-c1d6-46b0-8843-2d43506ab7d2]

I tried to compile Python 3000 under OpenBSD and the compilation fails because of a 'MemoryError':

Fatal Python error: can't create sys.path object : MemoryError() type : MemoryError refcount: 4 address : 0x20abfbd08 lost sys.stderr Abort trap (core dumped) *** Error code 134

Stop in /home/henry/compile/py3k (line 410 of Makefile).

The command which fail is: CC='gcc -pthread' LDSHARED='gcc -pthread -shared -fPIC ' OPT='-DNDEBUG -g -O3 -Wall -Wstrict-prototypes' ./python -E ./setup.py build

Here is the backtrace:

(gdb) r -E ./setup.py build Starting program: /home/henry/compile/py3k/python -E ./setup.py build Fatal Python error: can't create sys.path object : MemoryError() type : MemoryError refcount: 4 address : 0x2042d3d08 lost sys.stderr

Program received signal SIGABRT, Aborted. [Switching to process 20134, thread 0x2015d4800] 0x000000020dc4432a in kill () from /usr/lib/libc.so.48.0 (gdb) bt full

0 0x000000020dc4432a in kill () from /usr/lib/libc.so.48.0

No symbol table info available.

1 0x000000020dc8b105 in abort () at /usr/src/lib/libc/stdlib/abort.c:68

    p = (struct atexit \*) 0x2064fd000
    cleanup_called = 1
    mask = 4294967263

2 0x0000000000468a59 in Py_FatalError (msg=0x4ea6 \<Address 0x4ea6 out

of bounds>) at Python/pythonrun.c:1880 No locals.

3 0x000000000046e06c in PySys_SetPath (path=0x4ea6) at

Python/sysmodule.c:1390 v = (PyObject *) 0x0

4 0x0000000000466b8c in Py_InitializeEx (install_sigs=1) at

Python/pythonrun.c:213 interp = (PyInterpreterState *) 0x20f8af900 tstate = (PyThreadState *) 0x20adeda00 bimod = (PyObject *) 0x2042dc128 sysmod = (PyObject *) 0x2042dc128 pstderr = (PyObject *) 0x2042dc128 p = 0x0 codeset = 0x2042dc128 "\034"

5 0x0000000000474136 in Py_Main (argc=4, argv=0x20f0331a0) at

Modules/main.c:497 r1 = 0 r2 = 0 c = 0 sts = 4 command = 0x0 filename = (wchar_t *) 0x0 module = 0x0 fp = (FILE *) 0x964e70 p = 0x6c05 \<Address 0x6c05 out of bounds> unbuffered = 0 skipfirstline = 0 stdin_is_interactive = 1 help = 0 version = 0 saw_unbuffered_flag = 0 cf = {cf_flags = 0}

6 0x0000000000412866 in main (argc=4, argv=0x7f7ffffc7920) at

Modules/python.c:57 argsize = 140187732310304 argv_copy = (wchar_t **) 0x20f0331a0 argv_copy2 = (wchar_t **) 0x20f033140 i = 0 res = -231136 oldloc = 0x20e0c1b00 "C"

I also have core file. If you are interested mail me.

[fef5b636-c1d6-46b0-8843-2d43506ab7d2]

I forgot to mention, I made to following modification to configure.in so I could compile Python 3000 on OpenBSD 4.4

--- configure.in        (revision 66037)
+++ configure.in        (working copy)
@@ -250,7 +250,7 @@
   # On OpenBSD, select(2) is not available if _XOPEN_SOURCE is defined,
   # even though select is a POSIX function. Reported by J. Ribbens.
   # Reconfirmed for OpenBSD 3.3 by Zachary Hamm, for 3.4 by Jason Ish.
-  OpenBSD/2.* | OpenBSD/3.@<:@0123456789@:>@ | OpenBSD/4.@<:@0123@:>@) 
+  OpenBSD*)
     define_xopen_source=no
     # OpenBSD undoes our definition of __BSD_VISIBLE if _XOPEN_SOURCE is
     # also defined. This can be overridden by defining _BSD_SOURCE

[hyeshik]

This problem is due to OpenBSD's libc bug. It's fixed 3 days ago. (http://www.openbsd.org/cgi- bin/cvsweb/src/lib/libc/string/wcschr.c#rev1.4) We can workaround by replacing use of wcschr(ws, L'\0') to ws + wcslen(ws).

[fef5b636-c1d6-46b0-8843-2d43506ab7d2]

Indeed it looks like it is the source of the problem.

I created a patch to fix it.

But it looks like there is another problem, instead of crashing the Python interpreter goes into interactive mode instead of executing the 'setup.py' script ...

I don't think it is related, I have checked the result of ws = ws + wcslen(ws) and it seems to be correct.

I will investigate the problem and create another entry if it is unrelated.

[fef5b636-c1d6-46b0-8843-2d43506ab7d2]

Looks like my other issue is unrelated. It is caused by a buggy mbstowcs.

[fef5b636-c1d6-46b0-8843-2d43506ab7d2]

Here is a better patch which use the workaround only if wcschr is broken.

I was able to build the python interpreter and to run regrtest.py with it (some tests fail but it is very likely to be bugs in the modules)

[61337411-43fc-4a9c-b8d5-4060aede66d0]

Is there any problem with always computing the end of the string as "s + wcslen(s)"? I feel that this is actually more readable than wcschr.

Here is a patch that does that. It doesn't change PC/getpathp.c, since it's only used on Windows, where wcschr works correctly (AFAIK).

Henry, can you confirm that this patch still fixes the problem?

[fef5b636-c1d6-46b0-8843-2d43506ab7d2]

I just tested the patch and it fixes the problem.

[warsaw]

r66948