py2app 0.14: Command line argument handling is broken in python 3

[ronaldoussoren]

Original report by Paul McCarthy (Bitbucket: pauldmccarthy, GitHub: pauldmccarthy).

---

Howdy,

I have a sneaking suspicion that py2app's handling of command line arguments in python 3 is fundamentally broken.

My application is written to potentially accept a large number of command line arguments, and I regularly experience Bus error or Segmentation fault crashes when passing more than a handful of arguments to a py2app-built instance. I traced the crashes to py2app/src/apptemplate/main.c:1037-1053.

This block reads in command line arguments one-by-one, decodes them into UTF8 strings, and then casts them into an array which is then passed to PySys_SetArgv. However, there are a number of fundamental bugs in this block, related to invalid use of the Python c-api, and invalid type-casting.

The patch listed below fixes my problems for python 3, but breaks support for python 2. Unfortunately I don't know the py2app code well enough to propose a perfect solution to this problem. For example, I'm not sure how you would be able to typedef both the python-2 and -3 versions of the PySys_SetArgv function, nor how you will be able to use a single definition of the argv_new pointer for both py2 and 3.

--- ../ronaldoussoren-py2app-f9fd009cb9b1/py2app/apptemplate/src/main.c 2017-06-05 12:43:02.000000000 +0100
+++ py2app/apptemplate/src/main.c   2017-07-13 17:01:09.000000000 +0100
@@ -25,9 +25,9 @@
 typedef int (*PyRun_SimpleFilePtr)(FILE *, const char *);
 typedef void (*Py_FinalizePtr)(void);
 typedef PyObject *(*PySys_GetObjectPtr)(const char *);
-typedef int *(*PySys_SetArgvPtr)(int argc, char **argv);
+typedef int *(*PySys_SetArgvPtr)(int argc, wchar_t **argv);
 typedef PyObject *(*PyObject_GetAttrStringPtr)(PyObject *, const char *);
-typedef wchar_t* (*_Py_DecodeUTF8_surrogateescapePtr)(const char *s, ssize_t size);
+typedef wchar_t *(*Py_DecodeLocalePtr)(const char* arg, size_t *size);

 typedef CFTypeRef id;
@@ -858,7 +858,7 @@
     char buf[PATH_MAX];
     char c_pythonInterpreter[PATH_MAX];
     char c_mainScript[PATH_MAX];
-    char **argv_new;
+    wchar_t **argv_new;
     struct stat sb;
     void *py_dylib;
     int rval;
@@ -968,10 +968,10 @@
     LOOKUP(PySys_SetArgv);
     LOOKUP(PyObject_GetAttrString);
     LOOKUP(Py_BuildValue);
+    LOOKUP(Py_DecodeLocale);
 #if 0
     OPT_LOOKUP(Py_SetPath);
 #endif
-    OPT_LOOKUP(_Py_DecodeUTF8_surrogateescape);
     LOOKUP(PySys_SetObject);

@@ -1037,12 +1037,12 @@
     if (isPy3K) {
        int i;

-       argv_new = alloca((argc+1) * sizeof(wchar_t));
+       argv_new = alloca((argc+1) * sizeof(wchar_t*));
        argv_new[argc] = NULL;
-       argv_new[0] = (char*)py2app__Py_DecodeUTF8_surrogateescape(c_mainScript, strlen(c_mainScript));
+       argv_new[0] = py2app_Py_DecodeLocale(c_mainScript, NULL);

        for (i = 1; i < argc; i++) {
-     argv_new[i] = (char*)py2app__Py_DecodeUTF8_surrogateescape(argv[i], strlen(argv[i]));
+         argv_new[i] = py2app_Py_DecodeLocale(argv[i], NULL);
        }

     } else {

[ronaldoussoren]

This was fixed as a side effect of #370 . The patch above was correct, the call to surrogateescape was missing an argument (and is no longer needed now that there is a pu7blic API for this).