pthread_setcancelstate() uses invalid argument: PTHREAD_CANCEL_ASYNCHRONOUS

[dwjackson]

Expected behavior

pthread_setcancelstate() only takes either PTHREAD_CANCEL_ENABLE or PTHREAD_CANCEL_DISABLE whereas pthread_setcanceltype() takes either PTHREAD_CANCEL_DEFERRED or PTHREAD_CANCEL_ASYNCHRONOUS.

Actual behavior

In CoreDumpWriter.c, pthread_setcancelstate() is passed PTHREAD_CANCEL_ASYNCHRONOUS which is only valid as an argument to pthread_setcanceltype(). The reason this doesn't cause a run-time failure is that, on Linux, PTHREAD_CANCEL_ASYNCHRONOUS and PTHREAD_CANCEL_DISABLE are both defined in enums and both happen to have the same integer value. On other UNIX-like systems they can be (and are) defined differently. I believe that this means that the call is actually setting the cancel state to PTHREAD_CANCEL_DISABLE but given the context that doesn't seem like the intended behaviour.

• Legal values for pthread_setcancelstate() are shown here: http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_setcancelstate.html.
• The pthread.h header is in /usr/include on my system and contains the following:

enum
{
  PTHREAD_CANCEL_ENABLE,
#define PTHREAD_CANCEL_ENABLE   PTHREAD_CANCEL_ENABLE
  PTHREAD_CANCEL_DISABLE
#define PTHREAD_CANCEL_DISABLE  PTHREAD_CANCEL_DISABLE
};
enum
{
  PTHREAD_CANCEL_DEFERRED,
#define PTHREAD_CANCEL_DEFERRED PTHREAD_CANCEL_DEFERRED
  PTHREAD_CANCEL_ASYNCHRONOUS
#define PTHREAD_CANCEL_ASYNCHRONOUS     PTHREAD_CANCEL_ASYNCHRONOUS
};

Steps to reproduce the behavior

The erroneous use of pthread_setcancelstate() appears in CoreDumpWriter.c on line 88.

if(pthread_setcancelstate(PTHREAD_CANCEL_ASYNCHRONOUS, NULL) != 0){
    Log(error, INTERNAL_ERROR);
    Trace("WriteCoreDump: failed pthread_setcancelstate.");
    exit(-1);
}

Given the context, I think the correct call would be to pthread_setprocesstype() to "go back" from the DEFERRED setting on line 64.

System information (e.g., distro, kernel version, etc.)

Arch Linux Linux 4.18.16