goodboy / tractor

A distributed, structured concurrent runtime for Python (and friends)
GNU Affero General Public License v3.0
271 stars 12 forks source link

OS X: shared memory file name length limitation causing error #340

Open jaredgoldman opened 2 years ago

jaredgoldman commented 2 years ago

When running the test shm_test in this PR we kept running into the error OSError: [Errno 63] File name too long. After hard coding the filename to not include a UUID and be much shorter, the test succeeded. This left us puzzled as we weren't passing a gigantic string for a filename. However, after looking into the OSX Posix shm header files, we found that there's an arbitrary length of 31 characters for file names in the shared memory buffer here:

in posix_smh.h

#define _SYS_POSIX_SHM_H_

#include <sys/appleapiopts.h>
#include <sys/types.h>
#include <sys/proc.h>

struct label;

#define PSHMNAMLEN      31      /* maximum name segment length we bother with */

struct pshminfo {
    unsigned int pshm_flags;
    unsigned int pshm_usecount;
    off_t        pshm_length;
    mode_t       pshm_mode;
    uid_t        pshm_uid;
    gid_t        pshm_gid;
    char         pshm_name[PSHMNAMLEN + 1];
    void         *pshm_memobject;
    struct label *pshm_label;
};

#endif

Linked issues: https://github.com/pikers/piker/issues/306

Linked PRs https://github.com/goodboy/tractor/pull/338

Stackoverflow threads https://stackoverflow.com/questions/38049068/osx-shm-open-returns-enametoolong

To Do:

  1. Adjust tractor shm test to intercept operating system and shorten file name if needed
  2. Add change to Piker OR add full shm capability to Tractor OR ????
  3. Raise Python exception from Tractor linking to this issue
  4. Raise issue with Apple