semaphore errors on AIX 5.2

[cf2aeb3c-f43f-407a-8d37-30a2b5033222]

BPO	1234
Nosy	@malemburg, @tiran
PRs	python/cpython#831 python/cpython#552
Files	patch_aix_broken_semaphores.diff patch_aix_broken_semaphores.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/tiran' closed_at = created_at = labels = [] title = 'semaphore errors on AIX 5.2' updated_at = user = 'https://bugs.python.org/sable' ``` bugs.python.org fields: ```python activity = actor = 'dstufft' assignee = 'christian.heimes' closed = True closed_date = closer = 'christian.heimes' components = [] creation = creator = 'sable' dependencies = [] files = ['8485', '8486'] hgrepos = [] issue_num = 1234 keywords = ['patch'] message_count = 10.0 messages = ['56226', '56227', '56228', '57194', '57333', '57512', '57715', '59199', '61878', '61879'] nosy_count = 3.0 nosy_names = ['lemburg', 'sable', 'christian.heimes'] pr_nums = ['831', '552'] priority = 'normal' resolution = 'fixed' stage = None status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue1234' versions = ['Python 2.6', 'Python 2.5', 'Python 3.0'] ```

[cf2aeb3c-f43f-407a-8d37-30a2b5033222]

There used to be a bug open concerning this problem in the old bug tracker but I can't find it in the new bug tracker:

http://mail.python.org/pipermail/python-bugs-list/2005-June/029290.html

[cf2aeb3c-f43f-407a-8d37-30a2b5033222]

Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: The Written Word (Albert Chin) (tww-china) Assigned to: Nobody/Anonymous (nobody) Summary: semaphore errors from Python 2.3.x on AIX 5.2

Initial Comment: Newer versions of the AIX 5.2 libc implement POSIX semaphores and thus define _POSIX_SEMAPHORES in \<unistd.h>. However, when building Python 2.3.x (and I'm sure any other version with sem_init()), a few semaphore errors are encountered: building 'pcre' extension ./Modules/ld_so_aix gcc -bI:Modules/python.exp
-L/opt/TWWfsw/python232/lib/support -Wl,-brtl
-Wl,-blibpath:/opt/TWWfsw/python232/lib/support:/usr/lib
build/temp.aix-5.2-2.3/pcremodule.o build/temp.aix-5.2-2.3/pypcre.o -L/opt/TWWfsw/python232/lib -o build/lib.aix-5.2-2.3/pcre.so sem_trywait: Permission denied sem_wait: Permission denied sem_post: Permission denied

I ran the Python process under truss and saw that the sem_init() from Python/thread_pthread.h worked successfully. fork() was then called and a sem_trywait() operation (probably from PyThread_acquire_lock()) was called. From the sem_init() man page (http://publib.boulder.ibm.com/infocenter/pseries/index.jsp?topic=/com.ibm.aix.doc/libs/basetrf2/sem_init.htm): If the pshared parameter has a nonzero value, the semaphore is shared between processes. In this case, any process that can access the sem parameter can use it for performing sem_wait, sem_trywait, sem_post, and sem_destroy operations.

Only the sem parameter itself may be used for performing synchronization.

If the pshared parameter is zero, the semaphore is shared between threads of the process. Any thread in this process can use the sem parameter for performing sem_wait, sem_trywait, sem_post, and sem_destroy operations. The use of the semaphore by threads other than those created in the same process returns an error.

Setting the 2nd value of sem_init to 1 (was 0) solved the problem. However, I don't know if this is wise for security. Another solution is to set HAVE_BROKEN_POSIX_SEMAPHORES for "AIX/5" like for "SunOS/5.8".

----------------------------------------------------------------------

Comment By: Leanid (lim_1) Date: 2005-06-20 13:12

Message: Logged In: YES user_id=701470

I can confirm this problem. I am using scons to build my project on AIX. I could not run parallel builds (-j4) on AIX. Randomly, but very often xlC invocation would stuck and I have 4 xlCcode hanging and taking all CPUs.

After adding HAVE_BROKEN_POSIX_SEMAPHORES for AIX and rebuilding Python parallel builds on AIX are fine. Trying first fix with changing 0 to 1 in sem_init also resolved this problem, but I started to get some other unexplained error from scons, so had to deactivate semaphores all together.

Albert, thanks for this report, it helped a lot.

----------------------------------------------------------------------

[cf2aeb3c-f43f-407a-8d37-30a2b5033222]

I can confirm that this bug is still present in Python 2.4.1 (and probably later versions) on AIX 5.2 and 5.3.

When HAVE_BROKEN_POSIX_SEMAPHORES is not defined at compile time, Python will spend a lot of time in sem_wait, usually when acessing files. The problem does not always happen as it seems to depend on the type of file system in use.

In my programm running for 3m30s, Python would call about 400000 time sem_wait and spend about 2mins waiting. When Python is compiled with HAVE_BROKEN_POSIX_SEMAPHORES, the same programm will only take 1m30s.

I attach a patch which solves the problem.

[cf2aeb3c-f43f-407a-8d37-30a2b5033222]

The bug is still present in Python 2.5.1. The same patch applies. The patch is rather trivival, could someone please integrate it in trunk? Thanks in advance

[tiran]

I'm setting the priority to high because it sounds important for AIX users and the patch is *really* simple, just two additional lines for configure.in

[malemburg]

The problem is also present in Python 2.4 and 2.3. Confirmed on AIX 5.3.

[gvanrossum]

I have no way to test this.

[gvanrossum]

OK, accepted for 2.5 and 2.6.

Can someone with a stake and submit access (e.g. lemburg) check it in?

[malemburg]

Christian, could you apply this patch ? It needs a working autoconf 2.61 installation which I currently don't have.

The patch itself, does the trick, so should go in.

Thanks.

[tiran]

Fixed in r60464 (trunk) and r60465 (25)